Data Express

  • Publications
    Describes the Data Express publication set.
  • Readme
    Supplementary information on supported character encoding systems and data types.
  • Installation Guide
    Outlines the concepts and procedures needed to install Data Express.
  • Getting Started with Distributed Data Stores
  • Getting Started with z/OS Data Stores
    Contains explanatory information and a series of tutorials that show you how to start using the product in order to analyze and manage data stores from System z servers running z/OS.
  • Data Masking Guide
    Outlines the concepts and procedures used by the Data Masking module. Data Masking makes sensitive data anonymous in application test environments while maintaining the same characteristics as the original data.
  • Data Model Guide
    Provides description of the table structures used in the Data Builder, Data Masking, and Data Subset Extraction modules.
  • Data Subset Extraction Guide
    Outlines the concepts and procedures used by the Data Subset Extraction module. Data Subset Extraction uses selection criteria specified by the user to create a new test environment from an existing environment.
  • Error Messages
    Provides a detailed list of all Data Express warning, error, and informational messages displayed in log files during batch processing or on the screen during elaboration processing.
  • Front End Guide
    Outlines the concepts and procedures used by Data Express.
  • Life Cycle Guide
    Provides step-by-step instructions to help you handle modifications within the layout of the files already processed by Data Express.
  • Process for z/OS Guide
    Outlines the concepts and procedures used by Micro Focus Data Express for z/OS. It shows how to start and work with Data Express in the mainframe environment.
  • z/OS Toolkit Guide
    Outlines the concepts and procedures used by the Data Express Toolkit, which is a collection of utilities that help provide input to Data Express processes in order to automate data inventory functionality. It shows how to start and work with the toolkit in a mainframe environment.
  • Data Generation Guide
    This guide outlines the concepts and procedures used by the Data Express Data Generation module, which uses user-specified selection criteria, rules, and data to create a new test environment from scratch.
  • Glossary
  • Copyright


Publications

The following documentation is available for Data Express:

Data Masking Guide

This guide outlines the concepts and procedures used by the Data Masking module. Data Masking makes sensitive data anonymous in application test environments while maintaining the same characteristics as the original data.

Data Model Guide

This guide provides descriptions of the table structures used in the Data Builder, Data Masking, and Data Subset Extraction modules.

Data Subset Extraction Guide

This guide outlines the concepts and procedures used by the Data Subset Extraction module. Data Subset Extraction uses selection criteria specified by the user to create a new test environment from an existing environment.

Error Messages

This guide provides a detailed list of all warning, error, and informational messages that can be displayed in log files during batch processing or on the screen during elaboration processing.

Front End Guide

This guide outlines the concepts and procedures used by Data Express, a product that lets you analyze the database of a given application.

Getting Started with Distributed Data Stores

This guide contains explanatory information and a series of tutorials that show you how to start using Data Express. The tutorials introduce the major features in a number of short sessions, typically 10-30 minutes.

Getting Started with z/OS Data Stores

This guide contains explanatory information and a series of tutorials that show you how to start using Data Express in order to analyze and manage data stores from System z servers running z/OS. The tutorials introduce the major features in a number of short sessions, typically 10-30 minutes.

Installation Guide

This guide outlines the concepts and procedures needed to install Data Express.

Life Cycle Guide

This guide provides step-by-step instructions to help you handle modifications within the layout of the files already processed by Data Express.

Process for z/OS Guide

This guide outlines the concepts and procedures used by Data Express for z/OS. It shows how to start and work with Data Express in the mainframe environment.

Readme

Contains general information on the product including hardware and software requirements, backward compatibility, and known errors and restrictions.

z/OS Toolkit Guide

This guide outlines the concepts and procedures used by the Data Express Toolkit, which is a collection of utilities that help provide input to Data Express processes in order to automate data inventory functionality. It shows how to start and work with the Toolkit in a mainframe environment.

Parent topic: Data Express


Readme

General information on Data Express is available here. This information includes hardware and software requirements, backward compatibility, known errors and restrictions and so on.

If you install a WrapPack containing updates and fixes to your installation, information on the WrapPack is included here. However, last-minute information is only in the WrapPack Readme, which is downloadable with the WrapPack from the SupportLine Web site.

To find out how to get the latest information, see the chapter Updates and Product Support.

  • System Requirements
    This topic provides system requirements.
  • Installation Notes
    These notes provide information on installing Data Express.
  • New Features
    Describes the new features for Data Express 4.0 Update 24.
  • Updates and Product Support
    Describes how to obtain updates and receive product support.
  • Known Issues and Restrictions
    Describes known issues and restrictions with the current release.
  • Supplementary Information
    Describes supplementary information for this release.
  • Readme Help
    Describes the readme help document.
  • Contact Information
    Describes contact details for product support.
Parent topic: Data Express


System Requirements

This section provides system requirements.

  • Windows
    Describes system requirements for Windows.
  • z/OS
    Describes system requirements for Data Express for z/OS.
  • Extension Technology
    Describes the system requirements for the extension technology.
Parent topic: Readme


Windows

This section describes the Windows system requirements for using Data Express with Data Express for z/OS and Data Express for Distributed Systems solutions.

Hardware Requirements

The following minimum configuration for Windows is required:

Component | Description
Processor | Must be at least the minimum specified for the operating system you are using.
Memory
  • Data Express for z/OS: at least 512 MB is needed; 1 GB is recommended.
  • Data Express for Distributed Systems: at least 1 GB is needed; 1.5 GB is recommended.
Disk space
  • Data Express for z/OS: at least 30 MB.
  • Data Express for Distributed Systems: 300 MB of disk space available to install all the modules of the Data Express suite (500 MB recommended).
Video | Display with 800 x 600 SVGA in 256 colors. Preferred screen resolution 1024 x 768 or higher.

Software Requirements

The following software is supported:

Component | Details | Description
Operating Systems 32-bit Windows
  • Windows XP Professional SP2 or later
  • Windows Vista
  • Windows 7, 8, 10
  • Windows Server 2003 and 2003 R2
  • Windows Server 2008 and 2008 R2
64-bit Windows
  • Windows XP Professional SP2 or later
  • Windows Vista
  • Windows 7, 8, 10
  • Windows Server 2003 and 2003 R2
  • Windows Server 2008 and 2008 R2

When running on a 64-bit Windows platform, Data Express runs as a 32-bit application.

Third-party software | Data Express for Distributed Systems | Borland Database Engine. This software needs to be installed after the Data Express product installation process on Windows.
Data Express on z/OS

IBM DB2 Connect Personal Edition (or Data Server Client) needed to access Gateway DB2 Connect Enterprise Edition.

Borland Database Engine. This software needs to be installed after the Data Express product installation process on Windows.

Although not required, if you are using the z/OS solution we recommend that your Windows machine have a direct connection to the DB2 database through DB2 Connect.

User Authorization Requirements

For Data Express for z/OS, when a project is opened for the first time, the user ID and password for the connection to the database server where the work environment resides must be provided. The user ID provided must have read privileges on the following:

SYSIBM.SYSTABLES

All tables are created by Data Express during product installation.

Parent topic: System Requirements


z/OS

This section describes requirements for the System z servers running z/OS. If you are using Data Express for Distributed Systems, which means that your Knowledge Base will reside on a Windows machine, you can skip this section.

Hardware Requirements

Any mainframe system that supports the minimum software requirements listed in the Software Requirements section can be used. Data Express requires the following amount of disk space:

  • 183 MB disk space available for product installation
  • 3 MB of disk space per analyzed archive (50 fields)

Software Requirements

The following software is supported for z/OS:

  • MVS, OS/390 2.5 or later, or z/OS 1.1 or later
  • Language Environment 6.1 or later
  • IBM DB2 (z/OS version, or OS/390 version 8 or later)

DB2 Database Requirements

Data Express for z/OS uses a DB2 database that contains about 150 tables and views. The database must be at least DB2 V7R1 with proper DDF configuration.

The tablespace definition for one table and the DDL statements used to create the database and are shown below.

CREATE TABLESPACE      ANURDCLA
       IN              &URDBASE
       USING STOGROUP  &URSGROUP
       PRIQTY              $PQTY
       SECQTY              $SQTY
       FREEPAGE         $FPAGE
       PCTFREE            $PFREE
       BUFFERPOOL    &URBPTS
       SEGSIZE 64
       COMPRESS         $COMPRESS;
CREATE TABLE  &UROWNER..ANURDCLA
(MCRECID                SMALLINT             NOT NULL WITH DEFAULT,
 FILRECID                  INTEGER               NOT NULL WITH DEFAULT,
 FLDRECID                 INTEGER              NOT NULL WITH DEFAULT,
 CLATYPE                  SMALLINT            NOT NULL WITH DEFAULT,
 PRG                            SMALLINT            NOT NULL WITH DEFAULT,
 CDRULECLA            INTEGER               NOT NULL WITH DEFAULT,
 CONFID                     INTEGER               NOT NULL WITH DEFAULT,
 FLAGDEL                  CHAR      (001)      NOT NULL WITH DEFAULT,
 BESTCLA                   CHAR      (001)      NOT NULL WITH DEFAULT)
 IN  &URDBASE..ANURDCLA;

Data Set Requirements

Data Express for z/OS uses 45 partitioned data sets (PDS) and 4 sequential data sets (SEQ). PDS and SEQ data sets are identified by the combination of a prefix and a suffix. However, the user can only choose the prefix.

The current prefixes are MFDATA.VxRx for the VR release library, MFDATA.VxRxPTF for the PTF library, and MFDATA.VxRxPER for the Personalized library. These prefixes can be modified. Suffixes (for example, BIND, CLIST, and DDL) cannot be modified.

Data Express for z/OS uses the following 45 partitioned data sets:

MFDATA.V2R0.BIND
MFDATA.V2R0.CBL
MFDATA.V2R0.CLIST
MFDATA.V2R0.COPY
MFDATA.V2R0.DBRMLIB
MFDATA.V2R0.DDL
MFDATA.V2R0.DDLALTER
MFDATA.V2R0.FTOUTPUT
MFDATA.V2R0.JCL
MFDATA.V2R0.LOAD
MFDATA.V2R0.OBJ
MFDATA.V2R0.PACKAGE
MFDATA.V2R0.PANEL
MFDATA.V2R0.SKEL
MFDATA.V2R0.SYSPUNCH
MFDATA.V2R0PTF.BIND
MFDATA.V2R0PTF.CBL
MFDATA.V2R0PTF.CLIST
MFDATA.V2R0PTF.COPY
MFDATA.V2R0PTF.DBRMLIB
MFDATA.V2R0PTF.DDL
MFDATA.V2R0PTF.DDLALTER
MFDATA.V2R0PTF.FTOUTPUT
MFDATA.V2R0PTF.JCL
MFDATA.V2R0PTF.LOAD
MFDATA.V2R0PTF.OBJ
MFDATA.V2R0PTF.PACKAGE
MFDATA.V2R0PTF.PANEL
MFDATA.V2R0PTF.SKEL
MFDATA.V2R0PTF.SYSPUNCH
MFDATA.V2R0PER.BIND
MFDATA.V2R0PER.CBL
MFDATA.V2R0PER.CLIST
MFDATA.V2R0PER.COPY
MFDATA.V2R0PER.DBRMLIB
MFDATA.V2R0PER.DDL
MFDATA.V2R0PER.DDLALTER
MFDATA.V2R0PER.FTOUTPUT
MFDATA.V2R0PER.JCL
MFDATA.V2R0PER.LOAD
MFDATA.V2R0PER.OBJ
MFDATA.V2R0PER.PACKAGE
MFDATA.V2R0PER.PANEL
MFDATA.V2R0PER.SKEL
MFDATA.V2R0PER.SYSPUNCH

Data Express uses the following four sequential data sets:

MFDATA.V2R0.IMPSRC
MFDATA.KBPARAM
MFDATA.V2R0.ALLDDL
MFDATA.V2R0.ALLBNDPK

The installer can modify the names of the first and second qualifier of a data set (for example: MFDATA.V2R0, MFDATA.V2R0PTF, and MFDATA.V2R0PER) according to your requirements.

Note: The total space occupied by all data sets does not exceed 50 MB.

User Credential Requirements

Data Express for z/OS users must be suitably configured both at the TSO and DB2 level. For a proper configuration, the following authorizations are required:

  • RACF authorizations (create, update, and delete) to the product PDS and sequential files (for example, JCL, CLIST, and SKEL).
  • Authorizations to the creation of the product DB2 database (storage group and database).
  • Authorizations to the creation of the DB2 tablespaces, views, and indexes.
  • Authorizations to the execution of package and plan binds.

Site-specific Requirements

Most activities in Data Express for z/OS consist of batch jobs executed through JCLs, which are normally created by specific submission functions by updating predefined skeletons (parameters, environment, and database specifications).

Note: If necessary, you can create permanent copies of the updated JCLs and adjust them to your organizational requirements.

The following actions may be performed in order to customize the product to meet organizational standards:

  • Modify job names (function supported during product installation).
  • Customize the skeletons.
  • Start work with proprietary or third-party products for scheduling or controlling jobs.
  • Make adjustments as required to provide interfaces with scheduling products
    CREATE STOGROUP &URSGROUP VOLUMES (xxxxxxxx) VCAT xxx;
CREATE DATABASE &URDBASE STOGROUP &URSGROUP;
  • Modify names such as the DB2 storage group group (STOGROUP), database, and owner (function supported during product installation)
  • Modify names of tablespaces, tables, views, and indexes (function not supported during product installation) Define ALIAS or SYNONYM
  • During the installation phase at your site, the installer will need to know your standards for:
    • Jobs names, accounting, etc.
    • Names of each DB2 subsystem, name of the DB2 load library, name and plan of DSNTIAD, name and plan of DSNTIAUL.
    • Name of the volume on which to allocate your partitioned data sets.
Parent topic: System Requirements


Extension Technology

This section describes requirements needed for the Extension Technology: ODBC Extension or Oracle Extension.

Hardware Requirements

Component | Description
Processor | Must be at least the minimum specified for the operating system you are using.
Disk space
  • Optional ODBC Extension: less than 5 MB per platform, precise details provided during install.
  • Optional Oracle Extension: less than 5 MB per platform, precise details provided during install.

Software Requirements

Component Description
32-bit Windows
  • Windows XP Professional SP2 or later
  • Windows Vista
  • Windows 7, 8, 10
  • Windows Server 2003 and 2003 R2
  • Windows Server 2008 and 2008 R2
64-bit Windows
  • Windows XP Professional SP2 or later
  • Windows Vista
  • Windows 7, 8, 10
  • Windows Server 2003 and 2003 R2
  • Windows Server 2008 and 2008 R2
UNIX
  • IBM System p running AIX 5.3 or 6.1
  • IBM System z running SuSE Linux Enterprise Server 10 SP1
  • Itanium running HP/UX 11iv2 or 11iv3
  • x86 running Red Hat Linux 4.0 Update 1, 5.1, 5.3, 6.1, or 7.2
  • x86 running SuSE Linux Enterprise Server 10 SP1 or 11
  • SPARC running Solaris 9 or 10
Note: When running on 64-bit UNIX platforms, Data Express Extension works as a 32-bit application.

Run-time Knowledge Base Requirements

This section describes requirements that apply to both Windows and UNIX platforms.

Before using either the ODBC or Oracle Extension, a Run-time Knowledge Base needs to be created within your source data store(s). The Run-time Knowledge Base drives the operation and performance of the Data Express masking and subsetting processes.

The Run-time Knowledge Base consists of tables and indexes, and needs to be created using the schema name DEKB.

Note: As different data stores have different requirements for creating schemas and associated objects, Micro Focus strongly suggests that the database administrator (DBA) for each source data store create the associated Data Express Run-time Knowledge Base.

A script, createkb.sql is provided within the config directory. This script contains canonical DDL (Data Definition Language) statements, describing the Run-time Knowledge Base tables and indexes.

Please note that createkb.sql is not guaranteed to execute correctly for all source data stores as-is. Each data store can have slightly different syntax when it comes to CREATE TABLE and CREATE INDEX statements. The DBA for your source data store can assess what changes are necessary to allow the successful creation of the DEKB schema, and will have the appropriate permissions to create and populate that schema.

If you are the DBA for a source data store, please review the Customer Care Knowledge Base article #10277 for additional details on how to create the Run-time Knowledge Base for your source data store.

User Credential Requirements

This section describes requirements that apply to both Windows and UNIX platforms.

ODBC or Oracle Extension users are required to have the appropriate access to source and target data stores, as well as to the Run-time Knowledge Base, which is located in the source data store(s).

The table below lists the pertinent objects and specific privileges required to mask and subset data:

Object | Privilege Required
Source data store(s) | SELECT on tables to be processed within the specified schema(s).
Run-time Knowledge Base | INSERT, SELECT, and DELETE on the tables within the DEKB schema.
Target data store(s) | CREATE TABLE, INSERT, UPDATE, and DELETE on tables to be processed within the specified schema(s).
Note: As different data stores have different user credential requirements, Micro Focus strongly suggests that the DBA for the data stores listed above establish appropriate user credentials.

For security reasons, it may not be permissible to allow Data Express privileges to operate in the manner described in the table above. In this case, you can use Extension Technology in unload mode.

For more information regarding unload mode, see the chapter Extension Technology Utilities in the Getting Started with Distributed Data Stores guide.

ODBC Extension Requirements

To use the ODBC Extension with an ODBC-enabled data store, you will need:

  • ODBC driver relevant to your data store
  • The UNIX machine should be configured with the environment variable settings needed to provide connectivity to the ODBC data store. At minimum, this should include:
    • ODBCINI - this should be set to the ODBC configuration file LD_LIBRARY_PATH (Linux, Solaris), SHLIB_PATH (HP/UX), or LIBPATH (AIX) , for the ODBC Driver Manager.
    • Any additional environment variables as specified within the documentation of your ODBC Driver Manager and ODBC Driver software.

Data Express for Distributed Systems is regularly tested against the following data sources and ODBC drivers:

Data Source Version | ODBC Driver Name | ODBC Driver Version
Micro Focus XDB Server 6.0 | Micro Focus XDB for DX4.0 | 8.00.06.1001
Micro Focus MFXDBC | Micro Focus MFXDBC Driver | 8.5.0
Oracle 11g R1 | Oracle in OraDb11g_home1 | 01.00.06
Oracle 12 R1 | Oracle 12R1 |
Microsoft SQL Server | SQL Server | 85.1132.00
Microsoft SQL Server | SQL Server Native Client 10.0 | 100.2500.00
DB2 v8 | IBM DB2 ODBC Driver | 01.300.257
DB2 v9 | IBM DB2 ODBC Driver | 01.300.257
DB2 v10 | IBM DB2 ODBC Driver | 01.300.257
Sybase 6.0.0.4765 | Sybase Adaptive Server Enterprise | 05.00.1016
Informix 11.70 | IBM Informix ODBC Driver | 3.70 TC5

Oracle Extension Requirements

To use the Oracle Extension, the machine housing the Data Express installation requires, at minimum, an Oracle client installation, including the Oracle Call Interface (OCI) support, configured to connect to the Oracle data store.

Data Express supports the following Oracle versions:

  • 9i
  • 10g
  • 10g r2
  • 11g
  • 11g R2

The UNIX machine should be configured with the environment variable settings needed to connect to the Oracle database, for example:

ORACLE_HOME
ORACLE_SID
PATH
LD_LIBRARY_PATH (Linux, Solaris) SHLIB_PATH (HP/UX) or LIBPATH (AIX)
Parent topic: System Requirements


Installation Notes

After installing, you need to install the latest updates, from the Micro Focus product support Web site http://supportline.microfocus.com.

For installation instructions, refer to the following chapters of the Installation Guide, as appropriate:
  • z/OS - z/OS System Procedures.
  • Windows - Section Windows Installation of chapter Distributed Systems Procedures.
  • UNIX - Section UNIX/Linux Installation of chapter Distributed Systems Procedures.

Windows Installation Program

To install the product, you need to be logged in with a user-ID that has Administrator privileges on the local machine, so the installation software can set the environment appropriately.

The installation program should start automatically when you insert the CD. If after a few seconds the installation window hasn't appeared, run the installation program as follows: x:\setup.exe

where x is your CD drive.

If you do not install an option or a sub-option, you can add it later by using Control Panel > Add or Remove Programs and selecting Micro Focus Data Express 4.0 and clicking Change.

Offline Web Browser

To use your Web browser off line, you need the dial-up networking feature Windows XP installed. Otherwise you may have TCP/IP errors such as being unable find "localhost" or the numeric equivalent (127.0.0.1).

Uninstalling

To uninstall the product, you need to be logged in with the same user-ID as you used when you installed it.

To uninstall the product, you cannot simply delete its files from your hard disk. Instead, you uninstall by using Control Panel > Add or Remove Programs , on most Windows systems. You might need to ensure that Show Updates (at the top of the Add or Remove Programs dialog) is checked, so that any hot fixes or WrapPacks are listed.

When you uninstall, the only files deleted are those that the installation software installed. If your Data Express directory has not been removed, delete any unwanted files and subdirectories within it using Windows Explorer.

Repairing

If any product files, registry settings or shortcuts are accidentally removed at any point, you can perform a Repair on the installation to replace them. To do this, use Control Panel > Add or Remove Programs and follow the instructions on the screen.

Parent topic: Readme


New Features

Data Express version 4.0 Update 24 includes the following new features:

  • Incremental update of target data sources enhanced with additional options
  • Data store information copied more easily between different machines/companies and between different Data Express instances
  • Automatic creation of indexes and foreign keys in target data sources
  • Minor synthetic data generation improvements
  • Better management of RI constraints, providing deletion and recreation
  • Manage interfaces as tables in addition to standard MVS flat files
  • Enhanced management of NULL filters processing
  • Improved and enhanced logging provided for diverse scenarios
  • Minor batch life cycle improvements
Parent topic: Readme


Updates and Product Support

After you have installed Data Express, you need to apply the latest updates if there are any. These are located on the Micro Focus web site in the Support section.

Before you can access the updates, you need to register with Product Support, as follows:

  1. Go to the main product support page by entering http://supportline.microfocus.com in your browser.
  2. Click Register on the product support page.
  3. Enter your details. You will need your product serial numbers, which you can find on the colored card in the CD case or on a sticker on the box.

To access the updates:

  1. Either enter http://supportline.microfocus.com in your browser, or start your Micro Focus product and click Help > Micro Focus SupportLine.
  2. Log in.
  3. Click Product Updates in the left-hand menu column, and select the appropriate link.
  4. Apply the latest updates for your product and version number.

SupportLine is a Web-based support and information service from Micro Focus. It provides all the latest information and software updates for your development system:

  • Technical support information
  • Latest product information
  • Documentation updates
  • New and updated demonstrations and templates
  • Hints and tips
  • Lots more - try it and see!
Parent topic: Readme


Known Issues and Restrictions

The restrictions covered here are those that are not covered elsewhere in the documentation, or that are of a temporary nature and may be removed in a later product. Other restrictions are covered in the chapter Supplementary Information for Data Express:

  • Data Express does not support the INTERVAL DATE data type because the 32-bit Oracle Driver does not support it.
  • Data Express supports LOB field types for DB2 with some restrictions. For more information, see Getting Started with z/OS Data Stores, and the topic DB2 for z/OS Environment Restrictions in this book.
  • Data Express supports XML field types for DB2 with some restrictions. For more information, see Getting Started with z/OS Data Stores, and the topic DB2 for z/OS Environment Restrictions in this book.
  • Data Express supports VARBINARY fields types for DB2 with some restrictions. For more information, see the DB2 for z/OS Environment Restrictions topic and Getting Started with z/OS Data Stores.
  • Data Express does not support LOB field types for ADABAS databases. Also, some Binary field types are not handled during ADABAS FDT analysis.
  • For DB2 tables with non-direct access, Life Cycle requires SYSPUNCH and DCLGEN to be partitioned data set (PDS) members, even though the cataloging phase supports SYSPUNCH and DCLGEN as sequential files. If you want to use Life Cycle and you have cataloged SYSPUNCH in a sequential file, you will need to delete it from the knowledge base and re-catalog it as a PDS member.
  • The Extension Technology does not support use of the parallel execution mode where multiple methods access the same tables within the target data store(s).
  • Data Express does not support sampling on true binary data.
  • Currently Data Express only shows the first 1000 distinct values and ranges, which are sorted either numerically or alphanumerically based on the data element type. When there are more than 1000 distinct data element values, the Data Element Value ends at the 1000th unique value instead of the greatest data element value.
  • Data Express uses USERID and PASSWORD to connect to the databases containing lookup tables. The password is mandatory.
  • On 64-bit platforms, Data Express runs as a 32-bit application; therefore, you must use 32-bit ODBC connections for Distributed Loader and Distributed Exporter even when running Data Express on a 64-bit platform.
Parent topic: Readme


Supplementary Information

The following sections contain supplementary information about this release.

  • Supported Character Encoding Systems
    Describes the supported character encoding systems for this release.
  • Supported Data Types
    Describes the supported data types.
  • Data Type Restrictions
    Describes data type restrictions.
  • ADABAS Restrictions
    Describes restrictions for ADABAS.
  • Oracle Restrictions
    Describes Oracle nested tables and target requirement creation restrictions.
  • DB2 for z/OS Environment Restrictions
    Describes the DB2 for z/OS environment restrictions.
  • AS/400 Data Stores
    Describes support for AS/400 data stores.
Parent topic: Readme


Supported Character Encoding Systems

The following 1-byte character encoding systems are supported:

Character Encoding System | Windows CP | Mainframe CP | UNIX CS
ISO8859-1 Latin1 (Western Europe) | 1252 | *see table below | ISO8859-1(15)
ISO8859-2 Latin2 (Eastern Europe) | 1250 | IBM-870 | ISO8859-2
ISO8859-5 Cyrillic | 1251 | IBM-1025 | ISO8859-5* (iconv)
ISO8859-7 Greek | 1253 | IBM-875 | ISO8859-7
ISO8859-9 Turkish | 1254 | IBM-1026 | ISO8859-9
ISO8859-8 Hebrew | 1255 | IBM-424 | ISO8859-8
ISO8859-6 Arabic | 1256 | IBM-420 | ISO8859-6* (iconv)

The following mainframe character encoding systems are compatible with ISO8859-1:

Encoding System | Location
IBM-037(1140) - | USA, Canada, Australia, New Zealand, Netherlands, Brazil, Portugal
IBM-273(1141) - | Austria, Germany
IBM-274 - | Belgium
IBM-275 - | Brazil
IBM-277(1142) - | Denmark, Norway
IBM-278(1143) - | Finland, Sweden
IBM-280(1144) - | Italy
IBM-281 - | Japanese English
IBM-284(1145) - | Spanish
IBM-285(1146) - | United Kingdom
IBM-297(1147) - | France
IBM-500(1047) - | Latin-1
IBM-871(1149) - | Iceland

Data Express does not support any additional character encoding systems at this time. This includes any multi or mixed byte character sets for CJK languages, Vietnamese, or any of the various forms of UNICODE, including UTF-8, UTF-16, and UCS-2.

Parent topic: Supplementary Information


Supported Data Types

ODBC Extension

The ODBC extension works with the following data types:

  • SQL_SS_XML
  • SQL_GUID
  • SQL_BIT
  • SQL_TINYINT
  • SQL_BIGINT
  • SQL_LONGVARBINARY
  • SQL_VARBINARY
  • SQL_BINARY
  • SQL_LONGVARCHAR
  • SQL_CHAR
  • SQL_NUMERIC
  • SQL_DECIMAL
  • SQL_INTEGER
  • SQL_SMALLINT
  • SQL_FLOAT
  • SQL_REAL
  • SQL_DOUBLE
  • SQL_DATE
  • SQL_TIME
  • SQL_TIMESTAMP
  • SQL_VARCHAR
  • SQL_TYPE_DATE
  • SQL_TYPE_TIME
  • SQL_TYPE_TIMESTAMP

Oracle Extension

The Oracle Extension works with the following Oracle Built-in data types:

  • VARCHAR2
  • NUMBER
  • FLOAT
  • LONG
  • DATE
  • TIMESTAMP
  • INTERVAL YEAR TO MONTH
  • INTERVAL DAY TO SECOND
  • RAW
  • LONG RAW
  • CHAR
  • CLOB
  • BLOB
  • BFILE

The Oracle Extension also works with the following Oracle-supplied type: XMLType.

DB2 for z/OS Environment Data Type

Data Express for z/OS works with the following DB2 data types:

  • CHAR
  • VARCHAR
  • GRAPHIC (Processed as char)
  • VARGRAPHIC (Processed as Varchar)
  • TIME
  • TIMESTAMP
  • DATE
  • SMALL INTEGER
  • INTEGER
  • BIGINT
  • DECIMAL
  • NUMERIC
  • REAL FLOAT (Supported only for Loading process)
  • DOUBLE FLOAT (Supported only for Loading process)
  • DOUBLE PRECISION FLOAT (Supported only for Loading process)
  • DECFLOAT (Supported only for Loading process)
  • ROWID
  • LONGVARCHAR
  • LONGVARGRAPHIC
  • BINARY
  • VARBINARY

z/OS also works with the following Large Object data type:

  • CLOB
  • DBCLOB
  • BLOB
  • XML

COBOL & PL/I for z/OS Environment Data Type

Data Express for z/OS works with the following Cobol & PL/I data types:

  • CHAR PIC X(n)
  • BINARY PIC 9(n) COMP-4
  • PACKED PIC S9(p-s)V9(s) COMP-3
  • ZONED PIC S9(p-s)V9(s)
  • EDITED Use of edit char, e.g. PIC ZZ99

ADABAS for z/OS Environment Data Type

Data Express for z/OS works with the following ADABAS data types:

  • ALPHANUMERIC
  • BINARY
  • F LOATING POINT
  • PACKED DECIMAL
  • UNPACKED DECIMAL(ZONED)
Parent topic: Supplementary Information


Data Type Restrictions

This section lists size and format restrictions for data types.

Size Restrictions

The ODBC Extension has a size restriction of 1,048,576 bytes when processing the following data types:

  • IMAGE
  • TEXT
  • VARBINARY
  • VARBINARY(MAX)
  • VARCHAR(MAX)
  • XML

Format Restrictions

The ODBC Extension has a format restriction for field content that use the DATETIME or SMALLDATETIME data type for Data Subset Extraction: Data Subset Extraction rules must be specified using the format as documented by Microsoft.

Parent topic: Supplementary Information


ADABAS Restrictions

This section describes size restrictions for ADABAS data stores.

ADABAS Catalog Link

Data Express does not support a direct link to the ADABAS catalog . In order to mask or subset your data, you must first use the ADABAS FDT report to produce the output sequential file containing the structure definition of your ADABAS tables. Then, you must upload the decompressed files (that were created by the command ADACMP DECOMPRESS) that contain the content of your ADABAS tables.

For information about the ADABAS FDT report, see the chapter Load Sequential File with ADABAS FDT report in your Toolkit (z/OS) guide.

For information about the structure of the output sequential file, see the section ADABAS - API Load Sequential File in the chapter Sequential Files in the Data Model Guide.

Descriptors, Subdescriptors, and Superdescriptors

Descriptors are handled as normal fields, but subdescriptors cannot be handled. In order to mask or subset a subdescriptor, an exit routine that is associated to the field referenced by the subdescriptor is needed.

The superdescriptor clause in the FTD is not directly supported by Data Express. However, a superdescriptor can be handled if a combined data element is created in Data Builder that is composed of each field within that superdescriptor.

Target Environment Creation and Population Limitation

Sequential data sets that contain the output of the masking or subsetting process for ADABAS data stores are created by Data Express in the same way sequential data sets are created for other data store types. But, to populate your target environment, ADABAS utilities must be used as Data Express does not provide functionality that enables creation or population of your target environment.

Process Identifier Limitation

Data Express needs a specific Process Identifier in order to works properly with ADABAS data store.

We recommend using a pre-loaded Process Id ADAUN.

For information about the structure of the ADAUN Process Identifier, see the section Process Identifier and Databases Type Relationships in the chapter Getting Started with Data Builder in the Front End Guide.

Parent topic: Supplementary Information


Oracle Restrictions

Support for Oracle nested tables is not provided in this release of Data Express. If you are interested in obtaining an EAP release that supports nested tables, please contact Micro Focus SupportLine.
Parent topic: Supplementary Information


DB2 for z/OS Environment Restrictions

This section lists size restrictions for DB2 for z/OS Large Object data types like CLOB, DBCLOB, BLOB, XML, and for DB2 z/OS VARBINARY data types.

DB2 Large Object Data Types restrictions

Data Express does not support a direct link to all the DB2 Large Object data types for these three operations:

  • Masking
  • Subsetting
  • Sampling

In order to mask or subset or sample your data, you must first use file reference variables method to unload each LOB or XML to a separate file. With this method, the LOB or XML values are unloaded to a different file than the normal unload file. DB2 creates or uses a different output file for each LOB or XML value to be unloaded. The output file should be on the following type:

Member of a partitioned data set (PDS) or partitioned data set extended (PDSE). See the following example JCL :

//LOADLOB JOB KRM,MSGCLASS=A,CLASS=1,NOTIFY=&SYSUID
//* *******************************************
//* TABLE &OWNNAME.&TABNAME
//* (PDFBLO BLOB (27994))
//* (TXTCLO CLOB (16094) )
//* (PDFBLO BLOB (16032) )
//* UNIT(SYSDA) SPACE ((20,2) MB)
//* (PDFBLO VARCHAR(54) BLOBF TSYSLOB)
//* ****************************************
//STEP1 EXEC DSNUPROC,UTPROC=,SYSTEM=&SYSTEM,LIB=DSN810.SDSNLOAD
//SYSPRINT DD SYSOUT=*
//SYSIN DD *
  TEMPLATE TSYSPUN
     DSN('URADAR.&SS..&DB..PUNCH.&TABNAME')
     DISP(MOD,CATLG,CATLG)
  TEMPLATE TSYSREC
     DSN('URADAR.&SS..&DB..SYSREC.&TABNAME')
     DISP(MOD,CATLG,CATLG)
  TEMPLATE TSYSLOB
     DSN('URADAR.&SS..&DB..PDS.&TABNAME')
     DISP(MOD,CATLG,CATLG)
     DSNTYPE(PDS)
  UNLOAD DATA FROM TABLE &OWNNAME.&TABNAME
   (A, B,C VARCHAR(44) CLOBF TSYSLOB, D)
  UNLDDN(TSYSREC) PUNCHDDN(TSYSPUN)
/*
//

Data Express supports a direct link to all the Large Object data types for this operation:

  • Cataloguing
  • Life Cycle
  • Reload
  • Unload

For the Data Express skeleton of Unload and/or Reload's Job, there are some modifications to apply:

  • Add the "TEMPLATE" condition.
  • If needed, add the field list with the specific keyword &FLDLST to be substituted.

The following example shows the Data Express Unload skeleton with the modifications applied:

//SYSIN     DD *
  TEMPLATE TSYSLOB
     DSN('URADAR.&DBNAME.&TSNAME.PIPPO')
     DISP(MOD,CATLG,CATLG)
     DSNTYPE(PDS)
 UNLOAD FROM TABLE
             &OWNAME.&FILNAME   HEADER NONE    LIMIT 0
&FLDLST
&DBNAME.&TSNAME

Process Identifier Limitation

Data Express needs a specific Process Identifier in order to works properly with DB2 Large Object data type.

Since Data Express doesn't have a pre-loaded Process Id for this kind of data, we recommend creating a new Process Identifier following these specific properties:

  • Access Type: Unload File Access (UNLOAD)
  • I/O Program Name: KURFIO.

For information about the Process Identifier structure, see the section Process Identifier and Databases Type Relationships in the chapter Getting Started with Data Builder in the Front End Guide.

DB2 VARBINARY Data Types restrictions

When Data Express accesses DB2 by direct access, it does not work directly with the execution of sub-extraction query created when applying a method's filter criteria if the filter is applied to a VARBINARY column or to a combined field containing VARBINARY column.

In general, when Data Express accesses DB2 by direct access, it creates a SELECT * statement with a WHERE clause in order to apply the appropriate filters.

For VARBINARY data type, Data Express will not generate this WHERE condition, but just the SELECT * statement.

The access to data in this case is done by a sequential access and the application of filter's criteria is done by Data Express in a transparent way, applying the filter criteria to each record and discarding this record not satisfying the filter.

For information about the Filter properties, see the chapter Work with Method - Selection Class/Filter properties in the Data Subset Extraction Guide.

Parent topic: Supplementary Information


AS/400 Data Stores

There are several alternatives to mask and subset AS/400 data stores.

ODBC connection to AS/400

Access via ODBC to AS/400 data via ODBC driver has the following limitations:

  • DB2 connect 9 is needed in order to guarantee compatibility with AS/400 of Data Express.
  • Flat files created by the CRTPF command without DDLs cannot be accessed via ODBC. This access work if there is a DDL or if the file is created via SQL.
  • The target database needs commitment control and so it needs to be created and journaled before Data Express runs.
  • Performance is slow because each row moves from AS/400 to windows (where the engine runs), this is mainly true for masking process (while for subsetting this is mitigated by the fact the reduction criterion can use local AS/400 indexes, differently to other two alternatives).

The following native AS/400 column data types are supported (in parenthesis the corresponding SQL ones)

  • Character (character,varchar)
  • Packed decimal (decimal)
  • Zoned decimal (numeric)
  • Binary (integer, smallint, bigint)
  • Floating point (float, real, double)
  • Date
  • Time
  • Timestamp
  • Hexadecimal
  • Binary Character (binary)

The following native AS/400 column data types are not supported:

  • DBCS-Only
  • DBCS-Either
  • DBCS-Open
  • DBCS-Graphic (graphic, vargraphic, long vargraphic)

The following SQL column data types not having a correspondence with native AS/400 types are not supported:

  • long varchar
  • varbinary
  • decfloat
  • blob
  • clob
  • dbclob
  • datalink
  • rowid

Details about how Data Express deals with ODBC data stores are available in the documentation, in particular in Getting started with distributed data stores.

MVS flat files based approach

Engine can run on MVS. Flat files containing AS/400 data are handled as MVS sequential files. FTP is used in order to move data from AS/400 to MVS and vice versa. It is needed providing copybook mapping file layout.

Details about how Data Express deals with z-OS data stores are available in the documentation, in particular in Getting started with z-OS data stores.

Windows flat files based approach

Engine can run on windows. CPYTOIMPF is used in order to generate a delimited file, to be sent via FTP to a windows machine. Here it is processed via DataDirect ODBC 6.0 driver, provided by Progress. It is needed to provide a file structure in a native DataDirect format dictionary. Then masked data is sent via FTP back to AS/400, and restored with CPYFRMIMPF to target AS/400 table

Details about how Data Express deals with ODBC data stores (in this case with flat files) are available in the documentation, in particular in Getting started with distributed data stores.

Comparison between approaches

  • MVS flat file approach is preferred if an MVS is available. It requires copybooks, normally already available in the application
  • Windows flat file based approach requires more manual work than ODBC approach, and it is faster than it for masking and slower than it for subsetting (comparison of performances are strictly related to characteristics of the machines)
Parent topic: Supplementary Information


Readme Help

The Readme Help is a file containing extensive advice on possible problems and troubleshooting. The on-screen installation instructions include an opportunity for you to look at it. If you want, you can see the Readme Help even before starting to install Data Express.

Parent topic: Readme


Contact Information

If ever you have any problems or you would like additional technical information or advice, there are several resources.

Help with Problems

In some countries, product support from Micro Focus may be available only to customers who have maintenance agreements.

If you obtained this product directly from Micro Focus, contact us as described below. If you obtained it from another source, such as an authorized distributor, contact them for help first. If they are unable to help, contact us as described below.

Before you ask for help, try and find answers in the Product Information folder of the Help or in the main body of the Help.

Information We Need

However you contact us, please try to include the information below, if you have it. The more information you can give, the better product support can help you. But if you don't know all the answers, or you think some are irrelevant to your problem, please give whatever information you have.

  • The version of the product that you are using. You can locate the version number:
    • Windows: from the Help > About.
    • z/OS: from the Main Menu panel.
    • ODBC/Oracle Extension: from the banner displayed when executing the Extension.
  • Your operating system and version number.
  • The data stores you are working with, including client-side and server-side versions where appropriate. If using ODBC data stores, also include details of the vendor and version of the ODBC driver(s) being used, as well as the equivalent information about the RDBMS.
  • If using Data Express for z/OS, indicate whether you are using the Standard or Client/Server configuration.
  • An explanation of what you are trying to do and step-by-step instructions needed for reproducing the issue, including any error messages received.
  • Diagnostic information:
    • z/OS: Screenshots, log files, JCL scripts, output, and dumps
    • ODBC Extension/Oracle Extension: Contents of the config and log directories. We may request that you change the setting of the LogLevel parameter with the dxeconfig.cfg file to provide additional diagnostic information. See the chapter Using Extension Technology of the Getting Started with Distributed Data Stores for further details on the LogLevel parameter.
  • Details of the data store contents, for example:
    DDL, DCLGEN, or SYSPUNCH  (for SQL data stores)
Copybooks (SEQ, VSAM, GDG, DL/I)
FDT (ADABAS)
DBD (DL/I)

Contact Information

Our Web site gives up-to-date details of contact numbers and addresses. The product support pages contain considerable additional information, including the WebSync service, where you can download fixes and documentation updates. To connect, enter http://www.microfocus.com in your browser to go to the Micro Focus home page.

If you are a Micro Focus product support customer, please see your product support Handbook for contact information. You can download it from our web site or order it in printed form from your sales representative.

Support from Micro Focus may be available only to customers who have maintenance agreements.

Parent topic: Readme


Installation Guide

  • Who Should Read This Guide
    Describes the audience for the guide.
  • z/OS System Procedures
    Describes how to install on z/OS.
  • Distributed Systems Procedures
    Describes the procedures for installation on distributed systems.
Parent topic: Data Express


Who Should Read This Guide

This guide is for all users who are installing Data Express for z/OS or Data Express for Distributed Systems.

After installing, you need to install the latest updates from the Micro Focus SupportLine web site.

Parent topic: Installation Guide


z/OS System Procedures

This chapter describes the installation media and procedures needed to successfully install Data Express for z/OS. If you are installing Data Express for Distributed Systems, you can skip this chapter.

After installing, you need to install the latest updates from the Micro Focus SupportLine web site. See the topic Install Updates for more information.

  • Installation Media
    Describes the installation media.
  • Installation Phases
    Describes the installation phases.
  • New Software Installation Procedures
    Describes the new software installation procedures.
  • Enable Data Express for z/OS
    Describes how to enable the product.
  • Install Updates
    Describes the installation for this update.
  • Software Upgrade
    Describes the software upgrade process.
  • Toolkit Installation
    Describes the installation of the toolkit.
  • Installing Data Express Into Multiple DB2 Subsystems
  • Install PTFs
    Describes PFT Installation.
  • Customization Reference
Parent topic: Installation Guide


Installation Media

Data Express for z/OS is typically installed from a CD that contains:

  • IEUBPDTE - A JCL in ASCII format used to restore INSTLIB from a sequential data set.
  • All of the partitioned data set members, unloaded in a sequential (IEBUPDTE) format.
Note: If you downloaded Data Express from the web site or received it as an e-mail attachment, it is stored in a compressed ZIP file that contains the CD image. For the installation, assume the folder where the ZIP file has been uncompressed is the CD drive.
Parent topic: z/OS System Procedures


Installation Phases

The installation of the Data Express for z/OS module includes six main phases.

Software Setup

This phase creates and populates the libraries containing the installation procedure.

Parameters Selection

This phase allows the editing of the installation procedure parameters. In particular, the alias of the PDSs and sequential files (for example, CLIST, JCL, or DDL) where the product modules will reside is required.

In addition, the following DB2 related information is required:

  • DSN on which the database must be created.
  • Name of the storage group (already existing or to be created).
  • Name of the DB2 database to be created.
  • Name of the owner for the DB2 tables that will be created.
  • Name of the plan and collection to be created.
  • Names of the buffer pools for data and indexes.
  • Plan of the DB2 system DSNTIAD.
  • Plan of the DB2 system DSNTIAUL.
  • Names of the DB2 system libraries (RUNLIB and SDSNLOAD).

Data Set Libraries installation

This phase creates and populates the product libraries. The following requirements must be met:

  • RACF authorizations at the TSO level (create, update, delete) need to be provided to the product PDSs and sequential files.
  • The TCP/IP address of the mainframe must be specified where required.
  • The specified user must be authorized to transfer data from the PC to the mainframe.

User Profile Update

This phase updates the MVS user profile in order to properly work with the product.

Database Creation

This phase creates the product DB2 database.

Note: Skeleton names IURBNDPK and SINCRTAB needed for the bind phase, which can be changed to use an authorization policy different from the standard.
Note: The specified user must be authorized to create the product DB2 database (storage group and database), to create tablespaces, views, indexes, and to execute package and plan binds.

Database Population

This phase populates the product DB2 database.

Parent topic: z/OS System Procedures


New Software Installation Procedures

This section contains information regarding z/OS system installation procedures. Perform the procedures in this section if you are installing Data Express for z/OS for the first time.

Important: Before installing Data Express, create an empty directory on your local machine, and extract the files contained in zOS.zip, located on your installation media, into that directory. For the installation procedure, we refer to this newly created directory as your installation directory.

Follow the steps in the subsequent sections.

  • Allocate the Installation Sequential Data Set
    Describes how to allocate the installation sequential data set.
  • Copy JCL Files to the Mainframe
    Provides instructions for copying the iebupdte.jcl and iebupdtp.jcl files to the mainframe.
  • Copy the Installation Data Set to the Mainframe
    Describes how to copy the installation data set to the mainframe.
  • Allocate and Load the Installation PDS
    Describes how to allocate and load the installation PDS.
  • Set Installation Parameters
    Describes how to set required parameters.
  • Install the Data Set Libraries
    Describes how to install data set libraries.
  • Update the User Profile
    Describes the user profile udpate option.
  • Create the Database
    Describes how to create the database.
  • Populate the Database
    Describes how to populate the database.
Parent topic: z/OS System Procedures


Allocate the Installation Sequential Data Set

To allocate the installation sequential data set:

  1. On the mainframe, from the ISPF Primary Option Menu panel, select option Utilities.
  2. From the Utility Selection panel, select the option Data Set.
  3. In the Name field, type the name of an installation sequential data set (for example, MFDATA.INST.LIB.SEQ) to load the installation JCLs in IEBUPDTE format from your installation directory's \**\HostSide\Installation Procedure subdirectory.
  4. Specify option A to allocate the data set.
  5. On the Allocate New Set panel, enter the appropriate parameters.
    Note: The default Space units parameter is CYLS (cylinders).

    The Allocate New Data Set menu showing the current allocation parameters is displayed.

Once allocated, a Data set allocated message displays in the right corner.
Parent topic: New Software Installation Procedures


Copy JCL Files to the Mainframe

Here you copy the iebupdte.jcl and iebupdtp.jcl files from your installation directory's \**\HostSide\Installation Procedure subdirectory, in ASCII mode, into a member of a partitioned data set on the mainframe. To help you do this, we provide both FTP and Personal Communications (PC3270) assistance in the form of the following files, located in the same directory:
  • FTPINST.BAT - A batch command that executes the ftpinst.txt FTP script
  • ftpinst.txt - An FTP script executed by FTPINST.BAT
  • INST.SRL - A PC3270 transfer file list

Before using the provided FTP script or PC3270 transfer file list to transfer the files, you must customize them to include the correct values. See Customization Reference for specific customization details.

Use this procedure to transfer the files to the mainframe using the ftpinst.txt script:

  1. On your local machine, customize ftpinst.txt, located in your installation directory's \**\HostSide\Installation Procedure subdirectory, as specified in the FTP Script Customization topic.
  2. From a Windows command prompt, change to the installation directory's \**\HostSide\Installation Procedure subdirectory, and execute the following command:
    ftp -s:ftpinst.txt
    Note: Optionally, you can run the FTPINST.BAT file to execute the command. For details on customizing this file, see FTP BAT File (ftpput.bat) Customization .
Parent topic: New Software Installation Procedures


Copy the Installation Data Set to the Mainframe

  1. On your local machine, customize ftpputinstlib.txt, located in your installation directory's \**\HostSide\Installation Procedure subdirectory, as specified in the FTP Script Customization topic.
  2. Copy the customized ftpputinstlib.txt file into your installation directory\bin directory.
  3. From a Windows command prompt, change to the installation directory\bin directory and execute the following command:
    ftp -s:ftpputinstlib.txt
Parent topic: New Software Installation Procedures


Allocate and Load the Installation PDS

Define the installation PDS where the sequential data set is loaded.

  1. Using the ISPF editor, customize the IEBUPDTE member as specified in the JCL File (iebupdte.jcl) Customization topic.
  2. Submit IEBUPDTE to allocate and load the installation PDS.
  3. Browse the INST.LIB PDS.
  4. Select the member INSTLIB (not INST.LIB) using the Edit option.
  5. Change the first parameter in the CLIST INST.LIB (INSTLIB) to use the name of the installation PDS INST.LIB, and then execute the CLIST command file.

    For example, execute the TSO command:

    TSO EX '**.INST.LIB(INSTLIB)'
    Note: You can also execute the CLIST by typing ex on the line for the member INSTLIB.
The Product Installation Menu panel is displayed.
Parent topic: New Software Installation Procedures


Set Installation Parameters

  1. Select the Edit Product Installation Parameters option to display the Product Installation Parameters panel, where you can choose the name of the installation PDS and the parameter data set (that contains all of this information).
    Important:
    • The name of the library containing all the jobs to be submitted is created by putting together the Submitted Job Library Prefix (if it exists, this is an optional field), the user ID and the Submitted Job Library Suffix, all separated by a period (.).
    • The Parameter Data Set is a sequential file containing the environment parameters. The prefix of the name of this file must be different from the product library prefix. This file is also referred to as the KBPARM file. Be sure the file qualifier is one that you have authority to allocate.
    • At the end of the edit session, the Product Installation Menu panel displays again. Press F3 to exit the menu without submitting the jobs.
  2. On the Product Installation Menu panel, select option Edit CD Installation Parameters to display the CD Installation Parameter panel so that you can choose the sequential files (SEQ) prefix for file transfer. You must customize the Unit and Volume name, although the latter can be left blank for SMS managed DASD. The parameter Override PDS Prefix existence (N) prevents data set allocation if a data set exists with the same prefix. If you need to use an existing prefix, specify Y.
    Note: Press Enter to display the Product Installation Menu panel again.
  3. On the Product Installation Menu panel, select option Edit Knowledge Base Parameters to display the Knowledge Base Parameters panel so that you can insert name parameters such as DB2 and Language Environment PDS, DB2 storage group, database plan and collection, owner, and buffer pool.
  4. Insert parameter names into all fields as all parameters are mandatory:

    • Name of DB2 Subsystem (Dsn) – Name of the DB2 subsystem where the product tables will be created.

    • Name of Current SQLID – Name of the DB2 user who will perform the product tables creation.

    • Name of Micro Focus Data Plan – Name of the plan for the product.

    • Name of Micro Focus Data Collection – Name of the collection containing the product package.

    • Name of Micro Focus Data Storage Group – Name of the DB2 storage group where the product tables will be created.

    • Name of Micro Focus Data Database – Name of the DB2 database where the product tables will be created.

    • Name of Micro Focus Data Database Owner – Name of the DB2 creator that will be used in order to create all the product tables.

    • Name of Bufferpool for data and index – Names of the two buffer pools containing the data and the indexes of the product tables. They can contain the same value.

    • Name of DSNTIAD Program – Name of DSNTIAD program or equivalent.

    • Name of DSNTIAD Plan – Plan associated with DSNTIAD program or equivalent.

    • Name of DSNTIAUL Program – Name of DSNTIAUL program or equivalent.

    • Name of DSNTIAUL Plan – Plan associated with DSNTIAUL program or equivalent.

    • Dsname of DB2 Sdsnload – DB2 SDSNLOAD library.

    • Dsname of DB2 Runlib – DB2 RUNLIB library.

    Note: Press Enter to display the Product Installation Menu panel again.
  5. On the Product Installation Menu panel, select option Edit Knowledge Base Size Parameters to display the Knowledge Base Size Parameters panel so that you can customize size parameters such as the number of files, the number of fields for a single file, and the number of copies.

    Customize the following size parameters:

    • Files Number – Total estimated number of files that will be cataloged in the environment to be created. By files, we mean single DB2 or Oracle or SQL Server tables, single record formats of a flat file, single formats of a DL/I segment, or single Adabas files.

    • Fields Numbers (For File) – Average number of fields for each file.

    • Copy Number – Total estimated number of different copybooks I/O areas that will be cataloged in the environment to be created.

    • Fields Numbers (For Copy) – Average number of fields for copybook I/O areas.

    • Data Changer Files Number – Total estimated number of files that will be reduced or masked.

    • Data Changer Fields Numbers (For File) – Average number of fields for each file that will be reduced or masked.

  6. On the Product Installation Menu panel, select option Edit Job Card for Installation Jobs so that you can change the JOB cards of Data Express jobs.
    Note:
    • If you have line limits set in JES, then you should add /*JOBPARM LINES=999 or /*JOBPARM LINES = (999999,WARNING) to the JOBCARD.
    • To preview the job before submitting it, set TYPRUN=HOLD.
Parent topic: New Software Installation Procedures


Install the Data Set Libraries

This procedure describes how to install the data set libraries. After allocating PDS and SEQ files and the sequential file for FTP, transfer the Data Express sequential data sets to the mainframe so they can be used to populate the product installation PDS libraries. Once completed, you can then load Data Express.
To help you transfer the sequential data sets, we provide both FTP and Personal Communications (PC3270) assistance in the form of the following files. All reside in your installation directory's \**\HostSide\Installation Procedure subdirectory:
  • kbd.srl - A PC3270 transfer file list
  • ftpput.bat - A batch file that executes the ftpput.txt script
  • ftpput.txt - An FTP script that copies the sequential data sets to the mainframe

Before using the provided FTP script or PC3270 transfer file list to transfer the files, you must customize them to include the correct values. See Customization Reference for specific customization details.

  1. On the Product Installation Menu panel, select option Load Product from CD to display the Load Product from CD panel.
  2. Select PDS and SEQ product allocation to submit jobs to allocate the PDS and SEQ files that make up Data Express.
  3. Select Allocation of sequential file for FTP to submit a job to allocate a sequential file for file transfer.
  4. Customize the file ftpput.txt, located in your installation directory's \**\HostSide\Installation Procedure subdirectory, as specified in FTP Script (ftpput.txt) Customization.
  5. Copy the customized ftpput.txt file into your installation directory\bin directory.
  6. From a command prompt, change to the installation directory\bin directory and execute the following command:
    ftp –s:ftpput.txt
  7. Select Load Product from sequential files to submit a job to load Data Express from sequential files.
  8. From the ISPF Primary Option Menu panel, specify option 3.4 to manage data sets and to obtain a member list of the sequential files for the data set.
  9. Press PF11 and check the data set allocations to make sure all allocations are in less than 15 extents to be sure none failed to completely load.
  10. Press F3 to return to the Product Installation Menu panel.
  11. Select option Edit Job Cards for Micro Focus Data Jobs on the Product Installation Menu panel.
  12. Select option Actualize Parameter on the Product Installation Menu panel.
Parent topic: New Software Installation Procedures


Update the User Profile

When updating your user profiles, you must allocate Data Express PDSs to ISPF. For example: if your data set library prefix is PPP.GGG and the keylists of Data Express panels are contained in member URDPROF of PPP.GGG.JCL, you need to copy member URDPROF to ISP.SISPTENU or to UserPrefix.ISPF.ISPPROF.

To update user profiles:

  1. From the ISPF Primary Option Menu panel, specify option 3.4 to manage data sets, and then browse the installation JCL library.
  2. Next to the URDPROF member, type c.
  3. In the Name field of the To Other Data Set Name section, specify the library where your ISPF profile can be found.
  4. Exit your TSO session and start a new session.
  5. Execute the TSO command:
    TSO EXEC ‘PPP.GGG.CLIST(MFDATA)’
  6. Invoke the PFSHOW TAILOR command and ensure the parameters are set to 2, 2, 1, and 1 respectively.
  7. Save your changes.
Parent topic: New Software Installation Procedures


Create the Database

  1. On the Micro Focus Data - Main Menu panel, select option Environment Creation to display the Environment Creation panel.

    The Environment Creation panel contains the following options:

    • Linkedit All Modules - Do not execute this option in a standard installation where you have installed the loaded PDS from a sequential load data set. Submits jobs to linkedit all Data Express object modules.
    • Insert Environment Information - Inserts information about the environment and submits a job that results in the source member: PPP.GGG.DDL (DB2NREC).
    • Calculate Tables Information - Execute this option only if you are installing for the first time. Submits a job to calculate the values of parameters PRIQTY, SECQTY, FREEPAGE, PCTFREE, and COMPRESS for all tables used by Data Express.
    • Actualize All Tables - Execute this option only if you are installing for the first time. Submits a job to actualize the DB2 table sources used by Data Express.
    • Create All Tables - Execute this option only if you are installing for the first time. Creates the DB2 tables used by Data Express.
    • Create all Bind Packages and Plan JCL - Creates Data Express DB2 and JCL to create packages and plan.
    • Submit all Bind Packages and Plan JCL - Submits the JCL. The binding of all packages and plans is performed.
    • Load Message and Business Protection std tables - Submits a job to load messages and Business Protection standard tables. This process is part of Database Population.
    • Load Process Ids and Classes Tables - Submits a job to load the process IDs and classes tables. This process is part of Database Population.
  2. If you have installed the loaded PDS from a sequential load data set, do not execute this option. Only if instructed to do so by Micro Focus, select option Linkedit All Modules to linkedit all Data Express object modules. If you do linkedit all modules, exit your TSO session and start a new session before proceeding.
  3. Verify that the DB2 administrator has created a storage group and database, and has granted all the necessary authorizations to the database owner. A set of DDL samples creating those entities is provided with the product. The DDL to do this is IURDDSG and IURDDDB in the PDS PPP.GGG.DDL.
  4. On the Environment Creation panel, select the option Insert Environment Information to insert information about the environment and to submit job CALCREC that displays the source member: PPP.GGG.DDL (DB2NREC).
    Note: The source member contains the estimated number of records for each table and index.
  5. Note the estimated number of records for each table and index, and evaluate the size of the environment values.
    Note:
    • You must make your evaluation of the environment values as accurate as possible. DB2NREC is used in the following steps as a starting point in order to calculate the size of the tables. If the value in DB2NREC is too large, a space problem can result when creating tables later.
    • If the estimated number of records for a table or index is greater than 999,999, the batch job ends with return code "4," and a message containing the object name is issued in the job SYSOUT. If necessary, edit PPP.GGG.DDL (DB2NREC) to change the number of records before proceeding.
    • When you sample tables, the rules used to calculate the number of records assume that the sampling phase will be executed for all files. This is true if the environment is small, but if the environment contains thousands of files, the sampling could be executed only for some of the files, and the calculation criteria can change, so the actual number of records could be lower than the calculated one. This argument holds true for the following tables: ANURDLOG, ANURDSPE, and ANURDSPN (with their indices ANURDLOGX1, ANURDLOGX2, ANURDLOGX3, ANURDLOGX4, ANURDSPEX1, ANURDSPEX2, ANURDSPEX3, ANURDSPNX1, and ANURDSPNX2). A rule could be that all these files have the same number of records as ANURDFLF.
    • Execute this option only if you are installingData Express for the first time. If you are installing it to upgrade a previous release, see the topic Install Data Manager Updates
  6. On the Environment Creation panel, select the option Calculate Tables Information to submit the job IURCQTY.
  7. On the Environment Creation panel, select the option Actualize All Tables to submit the job SINACTAT.
  8. On the Environment Creation panel, select the option Create All Tables to submit the job SINCRTAT.
  9. On the Environment Creation panel, select the option Create all Bind Packages and Plan JCL to submit the job SINACTAB.
    Note: This job may return error messages at the end of the output JCL. If an error message is displayed at the end of the output JCL, note the Return Code.
  10. On the Environment Creation panel, select the option Submit all Bind Packages and Plan JCL to submit a job to execute the binding for all packages and JCL plans to submit the job SINCRTAB.
    Note: This job may return error messages at the end of the output JCL. If you receive a Return Code of 0 after executing this JCL and the JCL in the previous step, the job was successful and the error messages can be ignored.
Parent topic: New Software Installation Procedures


Populate the Database

Perform this procedure only if you are installing for the first time. If you are installing it to upgrade a previous release, see Upgrade Installation.

To populate the database:

  1. On the Environment Creation panel, select option Load Message and Business Protection std tables to submit job SURLPTB. (This is described in the previous section.)
  2. On the Environment Creation panel, select option Load Process Ids and Class Tables to submit job IURLOAST. (This is described in the previous section.)
The installation is now complete. Before using the product, you must enable it.
Parent topic: New Software Installation Procedures


Enable Data Express for z/OS

To use the fully enabled version of Data Express for z/OS, you must retrieve and specify the software key. This key is used to activate all mainframe Data Express functions appropriate for your product license.

If Data Express for z/OS is not fully enabled, some of the main batch jobs might terminate incorrectly.

The following topics describe the panels displayed by selecting options in the Product Enabling section of the Main Menu panel. Perform the procedure in each topic in the order specified:

  • Panels
    Describes enabling the panels.
Parent topic: z/OS System Procedures


Panels

This section describes the panels displayed by selecting options in the Product Enabling section of the Main Menu panel.

Note: If problems occur during product enabling, error messages might be displayed. For more information, see the Error Messages guide.
  • Get Request Key
    Describes the Get Request Key panel.
  • Apply Response Key
    Describes the Apply Response Key panel.
  • Display Enabled Options
    Describes how to view the enabled options.
Parent topic: Enable Data Express for z/OS


Get Request Key

The request key is the software key needed to activate Data Express for z/OS.

To get the request key:

  1. From the Main Menu panel, select option Get Request Key to display the Get Request Key (CEURGST) panel.
  2. Note the string listed in the Request Key field, and contact Micro Focus SupportLine to obtain the associated software key.
  3. Press F3 to return to the Main Menu panel.
Parent topic: Panels


Apply Response Key

After receiving the response key, you must apply it to activate Data Express for z/OS.

To apply the response key:

  1. From the Main Menu panel, select option Apply Response Key to display the Apply Response Key (CEURKEY) panel.
  2. Specify the software key you received from Micro Focus SupportLine.
  3. Press F3 to return to the Main Menu panel.
Parent topic: Panels


Display Enabled Options

After activation, you can verify the license status at any time.

To display enabled options:

  1. From the Main Menu panel, select option Display Enabled Options to display the Display Enabled Options panel.
  2. Note the information as appropriate.
  3. Press F3 to return to the Main Menu panel.
Parent topic: Panels


Install Updates

Data Express Updates can be found on the Micro Focus SupportLine web site as WebSyncs. Download the appropriate zip files and extract them into an installation directory so that they can be uploaded to the mainframe.

To install the Update:

  1. On the Micro Focus Data Express for z/OS - Main Menu panel, select option 24 to display the Installation New Modification Level - Main Menu panel.
  2. Select option Allocate Tempory PDS to submit the job SEQALC to create a temporary PDS to contain PTF objects. You must specify the first and second qualifier for the PDS and you must customize the Unit name and Volume name. However, the Volume name can be left blank for SMS managed DASD.
  3. Press Enter to return to the Installation New Modification Level - Main Menu panel.
  4. Select option Allocate Sequential File for PTF to submit the job FTPALC to create the working sequential files. You must specify the prefix to be used for the sequential files, and you must customize the Unit name and Volume name. However, the Volume name can be left blank for SMS managed DASDs.
  5. Press Enter to return to the Installation New Modification Level - Main Menu panel.
  6. Transfer the CD content into the previously created sequential files. First navigate to the installation directory containing the unzipped files and edit the file ftpput.txt so that each SEQ prefix is replaced with the fully qualified prefix for the data set name. Then from a command prompt, change to the installation directory and execute the following command:
    ftp -s:ftpput.txt
  7. Select option Load Temporary PDS from Sequential Files to submit the job LOAPTF to load the temporary PDS with PTF objects from the working sequential files. You must specify the first and second qualifier used to allocate the temporary PDS and the prefix used to allocate the sequential files created previously. To ensure a large enough limit for output lines, add the following parameter in the JOB statement: LINES=(999999,WARNING)
  8. Press Enter to return to the Installation New Modification Level - Main Menu panel.
  9. Select option Save old PDS of PTF to submit the job SAVPTF to save the PDS containing the latest PTF into the previously created temporary PDS. You must specify the same parameters as you did when executing the Allocate Temporary PDS option.
  10. Press Enter to return to the Installation New Modification Level - Main Menu panel.
  11. Select option Rename PDS to create a JCL with a new PDS containing the last PTF and to rename the previously created temporary PDS. You must specifiy the name of the temporary PDS used previously as the first parameter, and the name of the PDS that will contain the latest PTF as the second parameter.
  12. Exit Data Express.
  13. Exit the TSO session to release holds on data sets, and start a new session.
  14. Execute the JCL you created using the option Rename PDS (job RENPDS from your JCL library).
  15. Restart Data Express. On the Micro Focus Data Express for z/OS - Main Menu panel, select option 24 to display the Installation New Modification Level - Main Menu panel.
  16. On the Installation New Modification Level - Main Menu panel, select option Create all Alter Table (PTF) to submit the creation of alter table jobs for PTF libraries. When prompted, type R followed by the WebSync number for the current installation, which is usually one increment less than the number of the WebSync you are installing. For example, when installing Data Express 4.0 WebSync 23 on a system that currently runs WebSync 22, type R22.
  17. On the Installation New Modification Level - Main Menu panel, select option Submit all Alter Table (PTF) to submit the execution of alter table jobs for PTF libraries.
  18. On the Installation New Modification Level - Main Menu panel, select option Create all Bind Packages and Plan JCL (PTF) to submit the creation of bind jobs for PTF libraries.
  19. On the Installation New Modification Level - Main Menu panel, select option Submit all Bind Packages and Plan JCL (PTF) to submit the execution of bind jobs for PTF libraries.
  20. On the Installation New Modification Level - Main Menu panel, select option Create all Bind Packages and Plan JCL (PER) to submit the creation of bind jobs for PER libraries.
  21. On the Installation New Modification Level - Main Menu panel, select option Submit all Bind Packages and Plan JCL (PER) to submit the execution of bind jobs for PER libraries.
  22. Select option 14 Update Modification Level to execute a command to update the modification level. The RELEASE field on the Installation New Modification Level - Main Menu panel is updated.
Parent topic: z/OS System Procedures


Software Upgrade

Perform the following procedure only if you are installing Data Express for z/OS to upgrade a previous release.

A DB2 administrator must perform this procedure. To upgrade Data Express:

  1. From the chapter New Software Installation Procedures, perform the procedures described in each topic through Install Data Set Libraries.
  2. If you are migrating from a version earlier than 3.5, execute an Unload of the following tables: ANLICLOG, ANURDFLF, ANURDFXF, ANURDLOG, and ANURDSPE.
  3. On the Micro Focus Data - Main Menu panel, select option Environment Update to display the Environment Update panel.
  4. The Environment Update panel contains five options. You need to complete them in order:
    • Update All Tables - Submits a job for each changed or new table to create the DB2 tables used by the new release of Data Express, and to migrate the previous database instance. The release numbers for the tables you are updating are required. See the Data Model Guide for table information.
    • Load Message Tables - Submits a job to load the Data Express message tables.
    • Load Data Changer Tables - Submits a job to load the Data Changer Conversion tables. Note: Only execute the Load Data Changer Tables option if you are migrating a version older than V3R0.
    • Bind All Packages and Plan - Submits a job to execute the binding for all packages and JCL plans.
    • Update Level Id - Submits a job to update the Level ID for all sources. Before updating the Level ID, you have to delete the content of the HSLICLID table.
  5. If you are migrating from a version earlier than 3.5, execute a Load of the following tables: ANLICLOG, ANURDFLF, ANURDFXF, ANURDLOG, and ANURDSPE.
  6. On the Environment Creation panel, select option Create all Bind Packages and Plan JCL to create a job for a new DB2 package and JCL plan and then select option Submit all Bind Packages and Plan JCL to submit the job. 6.
  7. If you are migrating from a version earlier than V3R4, recompile all user-developed exit routines.
  8. If you are migrating from a version earlier than V3R5, use the Data Subset Extraction module to verify the numeric values for the filter types FILTER BY RANGE and FILTER BY LIST VALUE. These filters are now padded with 31 digits instead of 18 digits as in previous production versions. To correct the padding, open and reapply the filter.
    Note: For more information, see the section Filter Types in the Work with Method - Selection class/Filter properties chapter in the Data Subset Extraction Guide.
Parent topic: z/OS System Procedures


Toolkit Installation

To install the toolkit:
  1. Select option Toolkit Installation from the Main Menu panel. The Toolkit Installation - Main Menu panel contains eight options.
  2. Select option Allocate Temporary PDS to submit a job to create a temporary PDS to contain toolkit objects. You must specify the first and second qualifier for the PDS and you must customize the Unit name and Volume name. However, the Volume name can be left blank for SMS managed DASD.
  3. Press Enter to return to the Toolkit Installation - Main Menu panel. A job will be submitted.
  4. . Select option Allocate Sequential File for PER to submit a job to create the working sequential files. You must specify the prefix to be used for the sequential files, and you must customize the Unit name and Volume name. However, the Volume name can be left blank for SMS managed DASDs.
  5. Press Enter to return to the Toolkit Installation - Main Menu panel. The job FTPALC will be submitted.
  6. Transfer the CD content into the previously created sequential files. First navigate to the installation directory containing the unzipped files and edit the file ftpput.txt. Then from a command prompt, change to the installation directory and execute the following command:
    ftp –s:ftpput.txt
  7. Select option Load Temporary PDS from Sequential Files to submit the job LOAPTF to load the temporary PDS with toolkit objects from the working sequential files. You must specify the first and second qualifier used to allocate the tempory PDS and the prefix used to allocate the sequential files created previously.
  8. Press Enter to return to the Toolkit Installation - Main Menu panel. A job will be submitted.
  9. From the ISPF Primary Option Menu panel, browse a temporary library to ensure that PDS has been loaded.
  10. Select option Save old PDS of PER to submit a job to save the PDS containing the latest PTF into the previously created temporary PDS. You must specify the same parameters as you did when executing the Allocate Temporary PDS option.
  11. Press Enter to return to the Toolkit Installation - Main Menu panel.
  12. Select option Rename PDS to create a JCL with a new PDS containing the last version of the toolkit, and to rename the previously created temporary PDS. You must specifiy the name of the temporary PDS used previously as the first parameter and the name of the PDS that will contain the latest toolkit as the second parameter.
  13. Press Enter to return to the Toolkit Installation - Main Menu panel. The job RENPER will be stored in the Jobs data set for submission.
  14. After the job RENPER has finished executing, exit Data Express.
  15. Exit the TSO session to release holds on data sets, and start a new session.
  16. Execute the JCL job RENPER from the install JCL library.
  17. Restart Data Express, and select the option Toolkit Installation from Main Menu panel.
  18. On the Toolkit Installation - Main Menu panel, select option Bind Package (PER) to submit bind jobs.
  19. Select option Linkedit Module to submit linkedit jobs.
  20. Select option Create System Tables to submit the creation of the view on systables.
    CAUTION:
    Execute option Create System Tables only if you are installing Data Express for the first time. If you are installing it to upgrade a previous release, do not execute this option.
Parent topic: z/OS System Procedures


Installing Data Express Into Multiple DB2 Subsystems

This information outlines the concepts and procedures needed to migrate an existing Data Express installation to multiple DB2 subsystems.

  • Who Should Read This Information
  • Installation Cloning
Parent topic: z/OS System Procedures


Who Should Read This Information

This information is for all users who are cloning Data Express for z/OS.

Rad this information only if you need to clone Data Express environment from a DB2 subsystem where you need to complete the installation procedure to an empty DB2 subsystem.

The information is valid only if a new license key request is performed after performing the actions required for cloning.

If you did not apply latest updates before cloning, install the latest updates from the Micro Focus SupportLine web site.

See Installation Instructions for instructions on how to apply the updates

Parent topic: Installing Data Express Into Multiple DB2 Subsystems


Installation Cloning

This section describes the procedures needed to successfully clone Data Express for z/OS from DB2 subsystem to another empty DB2 subsystem.

  • Cloning Phases
  • Cloning a Database
    Describes database cloning for Data Express for z/OS modules.
  • Cloning Environment Libraries
    Details how to use and update a DB2 cloned environment for Data Express zOS installation cloning.
  • Enabling a New Cloned Environment
    Describes how to use the fully enabled version of Data Express for z/OS from a clone.
Parent topic: Installing Data Express Into Multiple DB2 Subsystems


Cloning Phases

Cloning Data Express for z/OS includes these main phases:
  • Cloning databases
  • Cloning environment libraries
  • Enabling the new cloned environment.
Parent topic: Installation Cloning


Cloning a Database

Note: This phase should be performed by the customer's DB Administrator.

This phase creates and populates the Knowledgebase tables containing only the structure of product environment already created.

To align with the existing environment, create all TABLESPACES, TABLE, VIEWs and INDEXES of the Data Express knowledge base in this phase.

To avoid further problems, DUPLICATE only the table structures, not the data already inserted.

You must copy these tables with their data.

  • HSURDMF
  • HSURDMFD
  • USDCHADD
  • USDCHCOM
  • USDCHNAM
  • USDCHSUR
  • HSURDCLA
  • HSURDPID
Parent topic: Installation Cloning


Cloning Environment Libraries

You can use and update the cloned environment either of the following methods depending on your need.

  • Multiple Set of Libraries - Use this method when requiring a separate and independent product installation to manage environments. This solution requires maintenance to be applied for each set of libraries and DB2 subsystem Websync files and scripts.
  • Single Set of Libraries - Use this method when you do not need to work in an independent manner and are using the same set of libraries for all the DB2 subsystem with the Data Express Knowledge base installed. This solution requires maintenance to a single set of libraries but multiple applications of Websync scripts for each DB2 subsystem.

Multiple Set of Libraries

To create different Data Express installations, you must copy the entire library set, change the names, and create a new PARAMETERs file.

  1. Copy all the libraries created at the end of the installation procedure, changing the names after copying by adding the text NEW to the end of the copy file string.

    For example, URADAR.DATA.* should be changed to URADAR.DATANEW.* when copied.

  2. Create a new PARAMETER FILE describing the new DB2 subsystem parameters. You must copy the original PARAMETER FILE into new one with a different name, then include it into new CLIST created under the newly created libraries.

    For example, URADAR.SEQ.KBPARAM should be copied to URADAR.SEQ.KBPARAM.NEW.

    Once copied, change the following DB2 related information included in the new parameter file.

    • DSN on which the cloned database was created
    • Name of the cloned storage group
    • Name of the cloned DB2 database
    • Name of the cloned owner of the cloned DB2 tables
    • Name of the cloned plan and cloned collection
    • Names of the cloned buffer pools for data and indexes
    • Plan of the cloned DB2 system DSNTIAD
    • Plan of the cloned DB2 system DSNTIAUL
    • Names of the DB2 system libraries (RUNLIB and SDSNLOAD)
    • Name of the new parameter file for the cloned environment.
  3. Check and modify the CLIST(MFDATA) to contain the references to new libraries.
          PROC 0 URPROJ  (URADAR)          +
      URGROUP (SV41)            +
      URGRPTF (SV41WRK)         +
      URGRPER (SV41PER)         +
      URSEQPAR (URADAR.SV41.SEQ.KBPARAM)
  4. Check and if needed modify the following skeletons containing job cards:
    • KURJBLIB
    • KURJBLPK
    • KURJBNAM
    • KURJBNPC
    • KURJBNPK
  5. At the end of the copying process, create and execute BIND packages. See Create the Database for information on creating and executing BIND packages.

Single Set of Libraries

To create a single Data Express installation managing different DB2 subsystems, you must create a new CLIST and PARAMETER FILE starting from the Data Express 4.0 existing libraries.

This phase allows the copying and the editing of the parameter's procedure file.

  1. Copy the original CLIST MFDATA into a new CLIST with a different name.

    For example: URADAR.DATA.CLIST(MFDATA)* should be copied to URADAR. DATA.CLIST(MFDATA2)

  2. Create a new PARAMETER FILE describing the new DB2 subsystem parameters. you must copy the original PARAMETER FILE into new one with different name, then include it into new CLIST created under the newly created libraries.

    For example, URADAR.SEQ.KBPARAM should be copied to URADAR.SEQ.KBPARAM.NEW.

  3. Once copied, change the following DB2 related information included in the new parameter file.
    • DSN on which the cloned database was created
    • Name of the cloned storage group
    • Name of the cloned DB2 database
    • Name of the cloned owner of the coned DB2 tables
    • Name of the cloned plan and cloned collection
    • Names of the cloned buffer pools for data and indexes
    • Plan of the cloned DB2 system DSNTIAD
    • Names of the DB2 system libraries (RUNLIB and SDSNLOAD)
    • Name of the new parameter file for the cloned environment
  4. At the end of the copying process, create and execute BIND packages. See Create the Database for information on creating and executing BIND packages.
Parent topic: Installation Cloning


Enabling a New Cloned Environment

To use the fully enabled version of Data Express for z/OS, you must retrieve and specify the software key. This key is used to activate all mainframe Data Express functions appropriate for your product license. If Data Express for z/OS is not fully enabled, some of the main batch jobs might terminate incorrectly.

This section describes the panels displayed by selecting options in the Product Enabling section of the Main Menu panel.

  • Panels
    Introduces the Panels section when Enabling a DB2 cloned environment for Data Express for z/OS.
  • Obtaining a Request Key
    Describes how to obtain a software key needed to activate Data Express for z/OS installed from a cloned environment.
  • Applying the Response Key
    After receiving the response key, you must apply it to activate Data Express for z/OS.
  • Displaying Enabled Options
    Describes Enabled licensing options for Data Express for z/OS installed from a cloned environment.
Parent topic: Installation Cloning


Panels

This section describes the panels displayed by selecting options in the Product Enabling section of the Main Menu panel.

Note: Error messages may display if problems occur during product enabling. See the Data Express Messages Guide for specific error message information.
Parent topic: Enabling a New Cloned Environment


Obtaining a Request Key

The request key is the software key needed to activate Data Express for z/OS. Perform the following steps to get the request key.

  1. From the Main Menu panel, select Get Request Key. The Get Request Key (CEURGST) panel appears.

  2. Note the string listed in the Request Key field, and contact Micro Focus SupportLine to obtain the associated software key.
  3. Press F3 to return to the Main Menu panel.
Parent topic: Enabling a New Cloned Environment


Applying the Response Key

ter receiving the response key, you must apply it to activate Data Express for z/OS. To apply the response key:

  1. From the Main Menu panel, select Apply Response Key. The Apply Response Key (CEURKEY) panel appears.
  2. Specify the software key you received from Micro Focus SupportLine.
  3. Press F3 to return to the Main Menu panel.
Parent topic: Enabling a New Cloned Environment


Displaying Enabled Options

After activation, you can verify the license status at any time.

  1. From the Main Menu panel, select Display Enabled Options. The Display Enabled Options panel appears.

  2. Note any license status information as appropriate and press F3 to return to the Main Menu panel.
Parent topic: Enabling a New Cloned Environment


Install PTFs

To install a PTF, use the installation instruction document for that PTF.

To begin your PTF installation:

  1. Restore the PTF objects in the PTF libraries.
  2. On the Main Menu panel, select option Apply PTF to display the Apply PTF panel. The Apply PTF panel contains 7 options.
    Note: Based on the instructions listed in your PFT installation document, the following options may or may not apply.
    • Linkedit Module - Submits a job to linkedit the specified module. After specifying the module name, press Enter to submit the job.
      Note: Use the Linkedit Module option for each object specified in the PTF installation instruction.
    • Duplicate Program - Submits a job to create a duplicate of objects for the specified module. After specifying the module name and number of objects to create, press Enter to submit the job. (For the number of objects to create, see the ReadMe of the HotFix you are installing.)
      Note: Use the Duplicate Program option for each object specified in the PTF installation instruction.
    • Create Table - Submits a job to create a DB2 table for an object specified in the PTF installation instruction. After specifying the table name, press Enter to submit the job.
      Note: Use the Create Table option for each object specified in the PTF installation instruction.
    • Update Table - Submits a job to update the new DB2 table specified in the PTF installation instruction. After specifying the table name, press Enter to submit the job.
      Note: Use the Update Table option for each object specified in the PTF installation instruction.
    • Load Message Tables - Submits a job to load message tables if specified in the PTF installation instruction. After specifying the table name, press Enter to submit the job.
    • Load Data Changer Tables - Submits a job to load data changer tables if specified in the PTF installation instruction. After specifying the table name, press Enter to submit the job.
    • Bind Package - Submits a job to bind a package for an object specified in the PTF installation instruction. After specifying the table name, press Enter to submit the job.
      Note: Use the Bind Package option for each object specified in the PTF installation instruction.
  3. Perform the detailed PTF install procedure specified in the installation instruction document in addition to the Linkedit Module and Bind Package steps, if required.
Parent topic: z/OS System Procedures


Customization Reference

  • FTP Script Customization
  • FTP Script (ftpput.txt) Customization
  • FTP BAT File (ftpput.bat) Customization
  • PC3270 Transfer File List (inst.srl) Customization
  • PC3270 Transfer File List (kbd.srl) Customization
  • JCL File (iebupdte.jcl) Customization
Parent topic: z/OS System Procedures


FTP Script Customization

Make the following changes:

Replace... | With your...
NNN.NNN.NNN.NNN | HOST IP ADDRESS
USER | TSO User ID
PSWD | User ID password
USERLIB.PDS | HOST JCL library
ENVID | HOST sequential installation file prefix

Sample ftpinst.txt script:

open 123.123.123.123
YOURUSERID
YOURPASSWORD
cd ..
bin
put iebupdte.jcl  'HLQ.SLQ.USERLIB(IEBUPDTE)'
disconnect
quit

Sample ftpputinstlib.txt script:

open 123.123.123.123
YOURUSERID
YOURPASSWORD
cd ..
bin
put instlib.bin  'HLQ.SLQ.MFDATA.INSTJB.LIB.SEQ'
disconnect
quit
Parent topic: Customization Reference


FTP Script (ftpput.txt) Customization

Make the following changes:

Replace... | With your...
NNN.NNN.NNN.NNN | HOST IP ADDRESS
USER | TSO User ID
PSWD | User ID password
SEQ | HOST sequential product files prefix
PRODUCT | HOST product files prefix

Sample ftpput.txt script:

open 123.123.132.123
YOURUSERID
YOURPASSWORD
cd ..
bin
put bind.bin          'MFI01.DE.V3R5M0.SEQ.bind'
put cbl.bin           'MFI01.DE.V3R5M0.SEQ.cbl'
put clist.bin         'MFI01.DE.V3R5M0.SEQ.clist'
put copy.bin          'MFI01.DE.V3R5M0.SEQ.copy'
put dbrmlib.bin       'MFI01.DE.V3R5M0.SEQ.dbrmlib'
put ddl.bin           'MFI01.DE.V3R5M0.SEQ.ddl'
put ddlalter.bin      'MFI01.DE.V3R5M0.SEQ.ddlalter'
put jcl.bin           'MFI01.DE.V3R5M0.SEQ.jcl'
put load.bin          'MFI01.DE.V3R5M0.SEQ.load'
put obj.bin           'MFI01.DE.V3R5M0.SEQ.obj'
put package.bin       'MFI01.DE.V3R5M0.SEQ.package'
put panel.bin         'MFI01.DE.V3R5M0.SEQ.panel'
put skel.bin          'MFI01.DE.V3R5M0.SEQ.skel'
put syspunch.bin      'MFI01.DE.V3R5M0.SEQ.syspunch'
put hsurdmf.bin        'MFI01.DE.V3R5M0.hsurdmf'
put hsurdmfd.bin       'MFI01.DE.V3R5M0.hsurdmfd'
put usdchadd.bin       'MFI01.DE.V3R5M0.usdchadd'
put usdchnam.bin       'MFI01.DE.V3R5M0.usdchnam'
put usdchsur.bin       'MFI01.DE.V3R5M0.usdchsur'
put usdchcom.bin       'MFI01.DE.V3R5M0.usdchcom'
disconnect
quit
Parent topic: Customization Reference


FTP BAT File (ftpput.bat) Customization

Change the drive designation, d:, to the drive letter that represents the drive that contains your Data Express 4.0 installation directory.

Change cd\bin to cd\2.2 installation directory\bin.

Parent topic: Customization Reference


PC3270 Transfer File List (inst.srl) Customization

Replace a: with the drive letter and path to your 4.0 installation directory; then make the following additional changes:

Replace... | With your...
USERLIB.PDS | HOST JCL library
ENVID | HOST sequential installation file prefix
Note: Be sure to use a TRANSFER-FILE TYPE named TEXT.
Parent topic: Customization Reference


PC3270 Transfer File List (kbd.srl) Customization

Replace d: with the drive letter and path to your 4.0 installation directory; then make the following additional changes:

Replace... | With your...
SEQ | HOST sequential product files prefix
PRODUCT | HOST product files prefix

Create a TRANSFER-FILE TYPE named 'BINARYDEF', similar to 'BYNARY' TYPE, but with DEFAULT specified in a file allocation parameter, and without any LRECL value.

Parent topic: Customization Reference


JCL File (iebupdte.jcl) Customization

Make the following changes:

Replace... | With your...
$PDSINST | Installation PDS name
$SEQINST | Installation sequential data set name
$UNIT | DASD unit name
$VOLSER | VOLUME name Note: For SMS managed DASD you must delete VOL=SER=$VOLSER from the JCL.

Sample iebupdte.jcl file:

//IEBUPDTE JOB (00001),MU,MSGCLASS=A,CLASS=A,NOTIFY=&SYSUID
//**********************************************************************
//* THIS JOB CREATES THE PDS FOR THE INSTALLATION PROCEDURE AND
//* LOADS IT FROM THE SEQUENTIAL FILE.
//*
//* YOU HAVE TO SPECIFY SOME INFORMATION.
//* TO DO SO, YOU HAVE TO CHANGE THE FOLLOWING WORDS IN THIS JOB:
//*
//* 1) CHANGE '$PDSINST' TO THE NAME OF THE INSTALLATION PDS
//* 2) CHANGE '$SEQINST' TO THE NAME OF THE INSTALLATION SEQ. FILE
//* 3) CHANGE '$UNIT'    TO THE NAME OF THE UNIT
//* 4) CHANGE '$VOLSER'  TO THE NAME OF THE VOLUME
//**********************************************************************
//STEP100  EXEC PGM=IEFBR14
//SYSPRINT DD SYSOUT=*
//PO       DD DSNAME=MFDATA.INST.LIB,
//            UNIT=SYSDA,DISP=(NEW,CATLG),
//            VOL=SER=8349082,SPACE=(TRK,(15,15,30)),
//            DCB=(RECFM=FB,LRECL=80,BLKSIZE=27920,DSORG=PO)
//*-----------------------------------------------------------------
//STEP200  EXEC PGM=IEBUPDTE,PARM=MOD
//SYSPRINT DD   SYSOUT=*
//SYSUT1   DD   DISP=OLD,DSN=MFDATA.INST.LIB
//SYSUT2   DD   DISP=OLD,DSN=MFDATA.INST.LIB
//SYSIN    DD   DISP=OLD,DSN=MFDATA.INST.LIB
Parent topic: Customization Reference


Distributed Systems Procedures

This chapter describes the installation media and procedures needed to successfully install Data Express.

After installing, you need to install the latest updates, from the Micro Focus SupportLine web site. See the section Updates and SupportLine for more information.

  • Windows Installation
    Describes how to install on Windows.
  • UNIX Installation
    Describes the UNIX installation.
Parent topic: Installation Guide


Windows Installation

To install, simply put the CD in your drive and follow the on-screen instructions that appear automatically after a few seconds. After installing, you can run Data Express from a button that appears at the end of the instructions and thereafter you can run it from your Windows Start menu.

  • Installation Media
    Describes the Windows installation media.
  • Software Setup
    Descibes the Windows software setup.
  • Software Update for Windows
    Describes how to obtain updates for Windows.
Parent topic: Distributed Systems Procedures


Installation Media

Data Express is typically installed from a CD that contains:

  • Windows client setup
  • XDB Knowledge Base
  • BDE software
  • ODBC and Oracle Extension software for UNIX and Windows
  • z/OS installation software

For z/OS, install the Windows client only. For Distributed Systems, install all components.

Note:
  • Your license determines which modules you are authorized to install and use.
    • Data Express for Distributed Systems
    • Data Express for z/OS
  • You must have administrator privileges on the PC for which you are installing.
Parent topic: Windows Installation


Software Setup

To begin your installation, follow the steps in the subsequent sections.

To use Data Express, you must request and specify the serial number. The serial number is the request key needed in order to activate all Data Express functions appropriate for your product license.

Note: The client-side serial number requested during your Data Express installation can only be obtained from Micro Focus SupportLine. You must provide the Micro Focus SupportLine representative the serial number specified on the CD box which is used as a request key in the Request Key (Serial Number) field.

To install Data Express:

  1. Before installing, contact Micro Focus SupportLine in order to get a Response Key by specifying the serial number you received from Micro Focus Distribution.
  2. Run Setup.exe from the installation CD. The InstallShield Wizard appears.

    The autorun starts. From the menu, install BDE if it is not installed already, or install Data Express directly.

    If you start DXP324000026.exe then the Windows Client installation Wizard starts directly.

  3. Click Next, accept the license agreement, and then click Next.
  4. In the User Name field, specify the appropriate name of the user or group of users who will use Data Express.
  5. In the Request Key (Serial Number) field, overwrite the text and specify your serial number
    Note: This value must be the same value you sent to Micro Focus SupportLine during your client side Response key request.
  6. In the Response Key field, enter the response key you received from Micro Focus SupportLine.
  7. Click Next to proceed with the install.
  8. If you are installing Data Express for z/OS, you must choose the product as appropriate (standard or client/server configuration). The architectural difference between the two configurations is explained in the Front End User Guide.
  9. Follow the prompts to complete your installation.
Parent topic: Windows Installation


Software Update for Windows

Software updates (WebSyncs or Hotfixes) are available on the SupportLine website ( http://supportline.microfocus.com ) at the Product Updates page. A link, pointing to an update web page, will be available for each WebSync or Hotfix.

For the windows component, the installation requires you to execute a .msp file that can be found in the mentioned web page.

After executing the file, the product version currently installed will be updated to the appropriate level of the WebSync/Hotfix.

A software update may require specific instructions. In this case, the SupportLine web page will contain any additional instruction to be followed in order to properly install the software update

Parent topic: Windows Installation


UNIX Installation

The Data Express ODBC and Oracle Extensions are installed from a .tar file that is provided as an electronic download in an e-mail message. Contained in the .tar file is the file install.txt, which provides further installation instructions.

  • Unix Software Updates
    Describes how to obtain updates for unix.
Parent topic: Distributed Systems Procedures


Unix Software Updates

Software updates (WebSyncs or Hotfixes) are available on the SupportLine website ( http://supportline.microfocus.com ) at the Product Updates page. A link, pointing to an update web page, will be available for each WebSync or Hotfix.

For the Unix component, applying a software update means downloading and installing a .tar file containing the new product extension version and replacing the existing installation.

A software update may need specific instructions. In this case, the SupportLine web page will contain any additional instructions to be follow.

Parent topic: UNIX Installation


Getting Started with Distributed Data Stores

This guide contains explanatory information and a series of tutorials that show you how to start using Data Express. The tutorials introduce the major features of Data Express in a number of short sessions, typically 10 - 30 minutes.

This guide is intended for customers who have the Data Express for Distributed Systems solution or for customers who have the Data Express for z/OS solution but have also purchased the additional ODBC Extension or Oracle Extension.

Note: Previous to the 4.0 release, the Data Express tool suite was intended for users who analyze and manage data stores from System z servers running z/OS. Now with the advent of Data Express 4.0, the suite of Data Express tools is available to non-mainframe users who can now analyze and manage data stores from Windows. In addition, Data Express 4.0 now provides support for distributed data stores for z/OS and Windows users, via the ODBC Extension and Oracle Extension.
  • Using Distributed Data Stores with Data Express
    Data Express 4.0 introduces the ability to mask and subset ODBC-enabled data stores. This new functionality basically mimics the same approach in methodology and functionality provided with the Oracle Extension Technology in previous version of Data Express.
  • Using Extension Technology
    This section of your Getting Started guide provides an overview of the Extension Technology, and provides detailed information on creating masking routines.
  • Data Express for Distributed Data Stores Tutorials
    These tutorials introduce the Data Express Windows client and show you how to configure your working environment and start and use the three Data Express modules: Data Builder, Data Masking, and Data Subset Extraction.
  • Oracle Alias Creation
    In order to use the Oracle Extension, you must first create an alias in the Borland Database Engine (BDE) for your source data store to communicate with the Oracle client installed on your machine. A BDE alias is not required for your target data store.
  • Oracle Wallet
    Describes Oracle Wallet support.
Parent topic: Data Express


Using Distributed Data Stores with Data Express

Data Express 4.0 introduces the ability to mask and subset ODBC-enabled data stores. This new functionality basically mimics the same approach in methodology and functionality provided with the Oracle Extension Technology in previous version of Data Express.

ODBC-enabled data stores are supported on both the z/OS and distributed platforms in the same way.

Note: The Oracle Extension workflow is the same as the ODBC Extension workflow with one exception: the input of multiple data stores for Data Masking and Data Subset Extraction is not supported.

Usernames and passwords are case sensitive and null passwords are allowed when connecting to a database.

The work-flow process for ODBC-enabled data stores looks like this:

  • Data Inventory Phase (Data Builder)
    • Load from Database (Distributed Loader)
    • Load from File (Distributed Loader)
  • Data Classification Phase (Data Builder)
    • Manual Association
    • Import Classification from Referential Integrity
    • Import Classification from Data Dictionary
    • Import Classification from Sampling Results
    • Limited Life Cycle Support
      • Skip, Overwrite, Update (Distributed Loader)
    • Data Analysis
      • Sampling
      • Fingerprinting
  • Data Masking Phase (Data Masking)
    • Metadata preparation
    • Export masking information (Distributed Exporter)
    • Actual masking (Extension Technology)
  • Data Subsetting Phase (Data Subset Extraction)
    • Method preparation
      • Create Referential Integrity methods
    • Export method information (Distributed Exporter)
    • Actual subsetting (Extension Technology )
    • Distributed Statistics Loader to enable run-time statistics
  • DBA Tasks for Extension Technology
    Describes DBA tasks required to access Data Express extension technology
  • Using Distributed Loader
  • Use of Data Analysis (Sampling) for Class Assignment
    Data Express for distributed data (both in the configuration with MVS knowledge base and in the configuration with XDB knowledge base) allows the capability of analyzing data content by using the sampling feature.
  • Importing Classes
    After you have completed the Data Inventory phase using the Distributed Loader, you need to import this information so that Data Express can create the corresponding classes. This is called the class generating process and is part of the Data Classification phase.
  • Creating Referential Integrity Methods
  • Using Distributed Exporter
    The Distributed Exporter utility is accessible from both Data Masking and Data Subset Extraction.
  • Using Distributed Sampler
Parent topic: Getting Started with Distributed Data Stores


DBA Tasks for Extension Technology

You can update your Run-Time Knowledge Base to enable Data Express extension technology. Depending on the current state of your Run-Time Knowledge Base, you can make the updates by executing one or more SQL scripts as specified in the topics listed below.

Script Execution

With regard to the execution of SQL scripts to make the updates, the following applies:
  • All scripts must be executed by a database user with DBA privileges.
  • By default, all scripts provided by Micro Focus grant privileges to PUBLIC. You can change this to limit privileges solely to the user ID(s) used to connect to your source data store(s) as dictated by your database security requirements.

User Credentials

For each database user to mask and subset data using the ODBC or Oracle Extension, you must grant access to source and target data stores as well as the Run-Time Knowledge Base, located in the source data store(s). The privileges required are:

Object | Required Privileges
Source data store(s) | SELECT on tables processed within the specified schema(s)
Run-Time Knowledge Base | INSERT, SELECT, and DELETE on tables within the DEKB schema
Target data store(s) | CREATE TABLE, INSERT, and DELETE on tables processed within the specified schema(s)
  • New Installations
    Describes DBA tasks required to access Extension Technology in a new installation of Data Express.
  • Data Express Upgrades
    Describes DBA tasks required to access Extension Technology after upgrading from Data Express version 4.0 or earlier.
  • Required Run-Time Knowledge Base Updates
    Information pertaining to updating the Run-Time Knowledge Base.
  • Centralized Run-Time Knowledge Base for the ODBC Extension
    Describes the requirements for configuring a centralized Run-Time Knowledge Base to use with the ODBC Extension.
Parent topic: Using Distributed Data Stores with Data Express


New Installations

Before using either the ODBC or Oracle Extension, a Run-Time Knowledge Base needs to be created within your source data store(s). The Run-Time Knowledge Base drives the operation and performance of Data Express' masking and subsetting processes. The Run-Time Knowledge Base consists of tables and indexes, and needs to be created using the schema name DEKB.

For convenience, we provide scripts that are tailored for working with data stores. To use these scripts, download the 3441.scripts.zip file from the DBA tasks for working with the Data Express Extension Technology article located in the Micro Focus Knowledge Base. This zip file contains individual scripts for the following data stores:
  • IBM DB2 LUW
  • Microsoft SQL Server
  • Oracle

If you are using a data store for which no specific script is provided, use the createkb.sql script, located by default in your Data Express installation directory, as the basis for the creation of the Run-Time Knowledge Base. This script contains "canonical" DDL (Data Definition Language) statements, describing the Run-Time Knowledge Base tables and indexes.

Before executing a script file:

IBM DB2 LUW
Edit the script file and update its CONNECT statement and connection credentials to connect to the appropriate database.
Microsoft SQL Server
Edit the script file and update its USE statement to specify the database in which to create and populate the Run-Time Knowledge Base.
Oracle
Create a database user named DEKB and grant the DEKB user all privileges required to create and populate the Run-Time Knowledge Base.
creatdb.sql
To ensure that createkb.sql executes correctly for all source data stores edit the file and, if necessary, update the CREATE TABLE and CREATE INDEX statements such that they enable the successful creation of the DEKB schema, and have the appropriate authority to create and populate that schema.
Parent topic: DBA Tasks for Extension Technology


Data Express Upgrades

When upgrading your installation from Data Express 4.0 or earlier, you must update the HSURDFIL table definition in all instances of the Run-Time Knowledge Base.

For convenience, we provide scripts that are tailored to update the HSURDFIL table definition in each Run-Time Knowledge Base. To use these scripts, download the 2045.AlterKB.zip file from the DBA tasks for working with the Data Express Extension Technology article located in the Micro Focus Knowledge Base. This zip file contains individual scripts for the following data stores:

  • IBM DB2 LUW
  • Microsoft SQL Server
  • Oracle
Parent topic: DBA Tasks for Extension Technology


Required Run-Time Knowledge Base Updates

Before installing any Data Express update, you must update your run-time tables by executing the following for each Run-Time Knowledge Base that uses the IBM DB2 LUW, Microsoft SQL Server, or Oracle data stores:
CREATE TABLE DEKB.HSLOGTAB(MSGCODE CHAR(7), FLAGENVI CHAR(1), ROUTNAME CHAR(8), VALRETCODE CHAR(2));
GRANT SELECT, UPDATE, DELETE, INSERT ON DEKB.HSLOGTAB TO PUBLIC;

For more information, see Logging Exit Routines in the Data Masking Guide.

If you are using a data store other than BM DB2 LUW, Microsoft SQL Server, or Oracle, update your run-time tables as required to allow updates to table definitions and grant the permissions required to update the DEKB schema.

Parent topic: DBA Tasks for Extension Technology
Related concepts
Logging Exit Routines


Centralized Run-Time Knowledge Base for the ODBC Extension

Note: This topic applies to the ODBC Extension only.

Because Data Express extension technology requires that the Run-Time Knowledge Base contains tables and indexes created using the DEKB schema, the DEKB schema must be in the source data store. If this requirement compromises production integrity when using the ODBC Extension, use a centralized Run-Time Knowledge Base.

To configure a centralized Run-Time Knowledge Base, you must create a new configuration file named hkblayer.ini and store it in the executables directory of the ODBC Extension installation (see Extension Technology Directories for more information). This configuration file defines the connection information required to access a centralized Run-Time Knowledge Base.

Include the following content in the hkblayer.ini file:

RDBMS = ODBC
DBNAME = ConnectionAlias
DBOWNER = TableOwner
USER = UserID
PSWD = Password
ConnectionAlias
The data store containing the Run-Time Knowledge Base tables. For the ODBC Extension, this is the data source name (DSN).
TableOwner
The table qualifier for the Run-Time Knowledge Base tables.
UserID and Password
The connection credentials required by the data store containing the Run-Time Knowledge Base tables.

For example:

RDBMS = ODBC
DBNAME = MYDSN
DBOWNER = MYOWNER
USER = MYUSER
PSWD = MYPASSWORD

After creating the hkblayer.ini file, start a command prompt and change to the executables directory that contains it. Enter the following command to generate the HKBLAYER.rc file required to access the centralized Run-Time Knowledge Base:

dxeencrypt hkblayer

Parent topic: DBA Tasks for Extension Technology


Using Distributed Loader

The Distributed Loader is accessible in Data Builder from [image].

For distributed data stores, the Distributed Loader is a utility that lets you load information from an ODBC-enabled database to the Knowledge Base as part of the Data Inventory process. The Knowledge Base can exist either on Windows or z/OS.

  • Loading Data Store Information
  • Limited Life Cycle Support
    For distributed data stores, both in z/OS and Distributed environments, Data Express provides limited Life Cycle support.
Parent topic: Using Distributed Data Stores with Data Express


Loading Data Store Information

Metadata information about your data stores can be loaded into Data Express in one of two ways: from a database or from flat files.

Not only can the Distributed Loader load information about your tables, columns, and indexes, but it can also work with the referential integrity information contained in your distributed data store.

  • Load from Database
    When you choose to load information from a database, the Distributed Loader provides a list of all the ODBC-enabled data stores defined in your Windows environment, and lets you select the data store's metadata information you want to load into the Data Express Knowledge Base.
  • Load from File
    Sometimes you may not be able to access your distributed data store from Windows. For this situation, the Distributed Loader provides the ability to load data store information from flat files.
Parent topic: Using Distributed Loader


Load from Database

When you choose to load information from a database, the Distributed Loader provides a list of all the ODBC-enabled data stores defined in your Windows environment, and lets you select the data store's metadata information you want to load into the Data Express Knowledge Base.

To load information about your data store from a database, begin by selecting the Load from Database tab.

Distributed Loader – Data Express for Distributed Systems

In Data Express for Distributed Systems, you will see an import checkbox for referential integrity labelled: Load Referential Integrity. This provides the ability to load your referential integrity information into the Windows Knowledge Base, in addition to standard schema information about your tables. You are now ready to import your referential integrity classes. See Importing Classes for more information.

The Distributed Loader utility differs slightly for the z/OS solution:

Distributed Loader – Data Express for z/OS

In Data Express for z/OS, an export checkbox labelled Export Referential Integrity is provided instead. In this environment, a file that contains referential information is created for you. Once this file is transferred to the mainframe, you can load and import your referential integrity classes. See Importing Classes for more information.

The ability to export referential integrity is based on an administrator's permission. This permission is provided by updating the ACTIVATION field in the HSURDRIA table in the knowledge base. By default, the table is empty. In order to allow the usage of referential integrity, the administrator needs to insert a value 'Y' in that field and to provide privileges to the user connecting to it.

Standard schema information about your distributed data store's tables is automatically loaded in the z/OS knowledge base. You can logically reduce the amount of information you wish to load by specifying the name of a schema in the Schema field. Otherwise, you leave this field blank and the Distributed Loader retrieves and loads all the information about all the schemas in the data store.

For more information about the Load from Database feature, review the Data Express for Distributed Data Stores Tutorials in this guide.

Parent topic: Loading Data Store Information


Load from File

Sometimes you may not be able to access your distributed data store from Windows. For this situation, the Distributed Loader provides the ability to load data store information from flat files.

To load information about your data store from a file, begin by selecting the Load from File tab. In the Distributed Systems solution, the Load from File tab of the Distributed Loader appears similar to the following:

Distributed Loader - Load from File (Distributed Systems)

In Data Express for Distributed Systems, two files can be used as input to the load process. The first file contains information about your tables, columns, and indexes. The second optional file contains information about your data store's referential integrity.

The Load from File tab of the Distributed Loader is different for the z/OS solution:

Distributed Loader - Load from File (z/OS)

In Data Express for z/OS, only the file that contains information about your tables, columns, and indexes can be used. For a z/OS Knowledge Base, your referential integrity information gets loaded into the Knowledge Base while executing the Import Classification from Referential Integrity job. See Importing Classes for more information.

The files used as input by the Distributed Loader are created by the Extension Technology utility: dxegenloadfile. This utility creates both files.

If you are using Data Express for z/OS, the referential integrity file created by dxegenloadfile is used as input to the Import Classification from Referential Integrity job.

If you are using the ODBC Extension or the Oracle Extension and want Data Express to load the user ID and password for the selected connection, run the dxegenloadfile using the -c parameter. This generates an input file with table information in the corresponding ODBC or Oracle log directory. See Extension Technology Directories for more information.

Draft comment:
Alberto: We don't have any documentation for using dxegenloadfile with parameters. Is -c the only one? We should have a reference topic for it. UPDATE: RESOLVED for now with additional dxegenloadfile content in the topic mentioned below.

For more information on dxegenloadfile, see Extension Technology Utilities in Using Extension Technology.

Parent topic: Loading Data Store Information


Limited Life Cycle Support

For distributed data stores, both in z/OS and Distributed environments, Data Express provides limited Life Cycle support.

After you have loaded information about your data stores in Data Express, your table definitions may have changed. For this possibility, the Distributed Loader provides the following options:

  • Skip - if the data store to be loaded already exists in the Knowledge Base, previously loaded tables are ignored.
  • Overwrite - if the data store to be loaded already exists in the Knowledge Base, old definitions are removed and the new definitions are inserted. If the data store is new, definitions are added. This option clears all the data element-class assignment in the existing data store.
  • Update - if the definition for the data store exists in the Knowledge Base, the following occurs:
    • Metadata is added to the Knowledge Base for new data elements in the data store.
    • Metadata for data elements that have different size values (length or precision or scale) are updated in the Knowledge Base, but information for assigned classes remains untouched.
    • Metadata for data elements that have different data types are updated in the Knowledge Base and all class assignments are cleared.
Parent topic: Using Distributed Loader


Use of Data Analysis (Sampling) for Class Assignment

Data Express for distributed data (both in the configuration with MVS knowledge base and in the configuration with XDB knowledge base) allows the capability of analyzing data content by using the sampling feature.

This feature helps in understanding the data element content and in assigning it to the appropriate class. The sampling process provides two types of results:

  • Standard sampling : it is a list of values with corresponding occurrences of the data elements selected for the process. This result can easily be checked via the user interface by human intervention.
  • Compressed sampling : this is a statistical distribution of the values of the data elements selected for the process, represented in a grid with the first character and number of characters of each value contained in the data element. The result is designed to be used by an algorithm based on class assignment by analogy (data elements having the same class are supposed to have the same statistical distribution of values).

The result of the sampling can be used to make class assignment in two different ways:

  • Manual assignment can be done by checking the standard sampling result and then by assigning classes to data elements through the "Work with data elements" feature. Note that the user interface is the same one of Data Express for MVS
  • Using assignment from sampling result. In this case, compressed sampling results are used. This process is further composed of two separate parts:
    • The requirement is defining one or more prototypes for each specific class, this happens in the "Work with data elements" feature by dragging a "Selected data element attribute" to a "Class sample". That is associating to a class the compressed sampling of a data element containing the same class.
    • Then an interactive process starts, and this process assigns the class to all data elements included in the parameters of the interactive process itself, basing the class assignment on the similarity of the compressed sampling results. This process compares the compressed sampling of the class to the compressed sampling of the data elements included in the parameters of the process itself, and if they fit it, assigns the class to the corresponding data element. Note that Data Express for MVS uses a similar but different one, based on the submission of a batch job from "Work with jobs" feature.

The sampling process will be described more in detail in the "Using Distributed Sampling" chapter, and it will be object of a tutorial described in the present manual.

The standard sampling and compressed sampling result verification is described in details in the "Work with data store" chapter of the Front End User Guide.

The manual class assignment and the prototype definition are described in details in the "Work with data elements" chapter of the Front End User Guide.

The assignment from sampling result is described in details in the present manual, in paragraph "Sampling result" of "Importing classes" chapter.

Parent topic: Using Distributed Data Stores with Data Express


Importing Classes

After you have completed the Data Inventory phase using the Distributed Loader, you need to import this information so that Data Express can create the corresponding classes. This is called the class generating process and is part of the Data Classification phase.

To begin the class generating process:

  1. From Data Builder, navigate to the Work with Data Elements window (click [image] or access it through Environment > Work with Data Elements).
  2. Click the Classes - Assignments tab.
  3. Click Import Class. The Import Class window is displayed.

Data Express offers three options at this point: you can either create classes for referential integrity (Referential Integrity), supply your own definitions of new classes for the columns in your tables (Data Dictionary), or you can create classes for related data elements after sampling.

Note: All options are available to the Distributed Systems solution and the z/OS solution when accessing distributed data stores. However, if you are using the z/OS solution for non-distributed (mainframe) data stores, sampling is handled by the Class Data Element Assignment and Data Store Data Element Sampling jobs. For more information about sampling with non-distributed z/OS data stores, see section Data Analysis in the Process Guide for z/OS .

  • Data Dictionary
    The Data Dictionary option gives you the ability to define your own classes for your distributed data stores from a flat file format.
  • Referential Integrity
  • Sampling Results
    The Sampling Results option gives you the ability to import your class assignments after you have assigned a fingerprint to your class. The class assignments are based on confidence values calculated using threshold recognition algorithms to determine the similarity of contents.
Parent topic: Using Distributed Data Stores with Data Express


Data Dictionary

The Data Dictionary option gives you the ability to define your own classes for your distributed data stores from a flat file format.

For the format of the Data Dictionary input file, see section Import Classification from Data Dictionary of the Data Model Guide .

For both the Distributed Systems and z/OS solutions, the Import Class window is the similar when Data Dictionary is selected.

For the Data Dictionary Import Type in the Distributed Systems solution, you simply need to provide the location (data store) of the Windows text file that contains your new class declarations, in addition to declaring the appropriate workspace (machine ID, company name). The classes are generated once you click Import.

For the Data Dictionary Import Type in the z/OS solution, you simply need to provide the location of the text file that contains your new class declarations (date store), in addition to declaring the appropriate workspace (machine ID, company name). Remember that this is a z/OS file specification. Select Import to schedule the Import Classification from Data Dictionary Job.

Parent topic: Importing Classes


Referential Integrity

For both the Distributed Systems and z/OS solutions, the Import Class window is similar when Referential Integrity is selected:

Import Class - Referential Integrity

For the Referential Integrity import type, you need to provide your workspace information (machine ID and company), distributed data store (database), and schema specification. The classes are automatically generated once you click Import. Select Referential Integrity if you want to create referential integrity classes.

For the Referential Integrity Import Type, you need to provide your workspace information (machine ID and company), distributed data store name (database), and additional fields provided in Data Store Qualification. Select Import to schedule the Import Classification from Referential Integrity job.

More information about the import classification jobs can be found in the Process for z/OS Guide , the Data Model Guide , and the Front End Guide .

Parent topic: Importing Classes


Sampling Results

The Sampling Results option gives you the ability to import your class assignments after you have assigned a fingerprint to your class. The class assignments are based on confidence values calculated using threshold recognition algorithms to determine the similarity of contents.

For this Import Type, you need to provide your workspace information (machine ID and company) and Content Threshold values. If desired, you can select other data store information to further restrict the class assignments. Select Import to assign classes to the data elements that have a fingerprint that matches the prototype fingerprint to the degree specified by the Content Threshold values.

Import Class - Sampling Results

For both the Distributed Systems and z/OS solutions, the Import Class window is the similar when Sampling Results is selected. However, it is only appropriate to import your classes in this way if your data stores are distributed (it works both with DB2 on MVS knowledge base and with XDB knowledge base).

Here is a summary of the parameters used by this function:

  • Machine ID - Identifier of the machine containing the data stores where Import Class process will work.
  • Company - Identifier of the company containing the data stores where Import Class process will work.
  • Data Store Qualification - With this set of selection parameters, it is possible to specify the scope of the class assignment process (which data stores you want to assign to the specified class).
  • Selected class - By using the Add botton it is possible to select one or more classes for the assignment process. This mean the process will attempt the assignation of the selected class or classes.
  • Super Classes - Selects a super class. This mean the process will attempt the assignment of the selected superclass.
  • Thresholds - Logical operations between Treshold1 and Treshold2. Data Express calculates two distinct formulas in order to understand how the processed column data distribution fits with the one of the prototype (by comparing compressed sampling result). Each one of the two formulas returns a number in the range 0-100, that is as higher as the two distributions are similar. By setting the two thresholds, and by setting the and/or parameter, it is possible to guide Data Express in the class assignment based on comparison between prototype and column to be assigned.

If you are working with z/OS data stores and a mainframe KB, you must schedule a series of jobs to perform the sampling functionality for you. For instance, you would schedule the job Data Store Data Element Sampling to select the sampling configuration, schedule the job Class Data Element Assignment to suggest class assignments, and then confirm the class assignments in the Work with Data Elements window of Data Builder.

For more information about assigning classes for your z/OS data stores, see section Data Analysis of chapter Project Implementation in the Process for z/OS Guide

For more information about importing classes by sampling results for distributed data stores, see the Simple Sampling exercise in this guide.

Parent topic: Importing Classes


Creating Referential Integrity Methods

After you have imported your referential integrity classes, you can create subsetting methods that adhere to those specifications.

  • Data Express for z/OS
  • Data Express for Distributed Systems
    In Data Express for Distributed Systems, from Data Subset Extraction, you can create a Referential Integrity method automatically.
Parent topic: Using Distributed Data Stores with Data Express


Data Express for z/OS

In Data Express for z/OS, you will need to schedule the Import Method from Referential Integrity Job. See the Process for z/OS Guide and the Front End Guide for more details.

Parent topic: Creating Referential Integrity Methods


Data Express for Distributed Systems

In Data Express for Distributed Systems, from Data Subset Extraction, you can create a Referential Integrity method automatically.

Click [image] (Import Method from Referential Integrity) to access the Import Method window:

Referential Integrity - Import Method

After supplying your workspace and method group, you need to supply the new method's name and description. In addition, you must also supply the class that drives the referential integrity (the leading class), and the distributed data store.

After selecting Import, the Referential Integrity method is created.

For more information, review the Data Express for Distributed Data Stores Tutorials in this guide.

Parent topic: Creating Referential Integrity Methods


Using Distributed Exporter

The Distributed Exporter utility is accessible from both Data Masking and Data Subset Extraction.

Its role is reasonably straight-forward; it creates and provides necessary information to the Extension Technology so that actual masking and subsetting of your distributed data store can occur.

There are two areas of interest when it comes to using the Distributed Exporter with distributed data stores:

  • If you use ODBC-enabled data stores, you can mask and subset across all your data stores with one invocation of the ODBC Extension. This feature is not available with the Oracle Extension.
  • You also have the ability to subset and mask within one distributed data store, as long as a distinct target schema name is provided.

For both Data Masking and Data Subset Extraction, the information you need to provide are very similar. For both scenarios you need to provide:

  • your workspace (machine ID and company)
  • your source and target distributed data store credentials
  • directory where the output files for the Extension Technology should be placed

For Data Subset Extraction, you can only export one method at a time by design.

  • Data Masking
  • Data Subset Extraction
  • SQL Files Generated by Distributed Exporter
Parent topic: Using Distributed Data Stores with Data Express


Data Masking

To access the Distributed Exporter, click [image] from the main window of Data Masking.

To export from Data Masking, you need to provide the following input:

  • Your workspace (machine ID, company)
  • Your source data store credentials (source database, source schema, source user ID/password)
    Note: The source database is an ODBC DSN configured to access that data store.
  • Your target data store credentials (target database, target scheme, target user ID/password)
    Note: The target database is an ODBC DSN configured to access that data store. The Extension technology does not support case sensitive object names.
  • Directory where you want to place output files for Extension Technology (Output directory)
Note: If the source database and the target database are the same, the schemas cannot be the same.

In addition, if you are using ODBC-enabled data stores, you can provide multiple sets of source and target data store credentials.

After you have successfully exported from Data masking, the following files are created in your output directory:

  • method.rc - Encoded connection criteria for each data store referenced.
  • elab.txt - List of data stores and the enabled columns within those data stores that need to be masked.
  • CREATETABLE.sql - list of the tables that need to be masked; a generic CREATE TABLE statement is provided for each table.
  • CREATEINDEX.sql - list of all the indexes for the tables that need to be masked; a generic CREATE INDEX statement is provided for each index.
  • ALTERTABLE_RI.sql - list of all the primary and foreign keys for the tables that need to be masked; a generic ALTER TABLE statement is provided for appropriate primary and foreign keys.
Note: SQL files can be used while utilizing the Extension Technology; see section SQL Generated by Distributed Exporter for more details.
Parent topic: Using Distributed Exporter


Data Subset Extraction

To access the Distributed Exporter, click [image] from the main window of Data Subset Extraction.

To export a method from Data Subset Extraction, you need to provide the following input:

  • Your workspace (machine ID, company)
  • Your method (group, method)
  • Your source data store credentials (source database, source schema, source user ID/password)
    Note: The source database is an ODBC DSN configured to access that data store.
  • Your target data store credentials (target database, target schema, target user ID/password)
    Note: The target database is an ODBC DSN configured to access that data store. The Extension technology does not support case sensitive object names.
  • Directory where you want to place output files for Extension Technology (Output directory)
Note: If the source database and the target database are the same, the schemas cannot be the same.

In addition, if you are using ODBC-enabled data stores, you can provide multiple sets of source and target data store credentials.

After you have successfully exported from Data Subset Extraction, the following files are created in your output directory:

  • method.rc - Encoded connection criteria for each data store referenced.
  • method.txt - Information identifying the method.
  • filter.txt - Information about the filters used in the method.
  • elab.txt - Information about the elaborations in the method.
  • cbfld.txt - Information about combined data elements used in the method.
  • CREATETABLE.sql - list of the tables that need to be masked; a generic CREATE TABLE statement is provided for each table.
  • CREATEINDEX.sql - list of all the indexes for the tables that need to be masked; a generic CREATE INDEX statement is provided for each index.
  • ALTERTABLE_RI.sql - list of all the primary and foreign keys for the tables that need to be masked; a generic ALTER TABLE statement is provided for appropriate primary and foreign keys.
Note: SQL files can be used while utilizing the Extension Technology; see section SQL Generated by Distributed Exporter for more details.
Parent topic: Using Distributed Exporter


SQL Files Generated by Distributed Exporter

Whenever you use the Distributed Exporter to export masking or subsetting information for consumption by the Extension Technology, three SQL files are created. These three files (CREATETABLE.sql, CREATEINDEX.sql, and ALTERTABLE_RI.sql) contain "canonical" DDL (Data Definition Language) statements about the source distributed data store's tables, indexes, and primary/foreign keys.

Normally, the Extension Technology will create tables in the target distributed data store with the same table specifications as in the source; however, no indexes or referential integrity constraints are created. If the Extension Technology "sees" that the tables already exist, it simply places the masked or subsetted data into those already created tables.

If you want your target distributed data store to more closely mimic the schema of your source distributed data store, these SQL files can help you in that effort.

Note: The SQL files are not guaranteed to execute properly with all ODBC data stores. Each ODBC data store can have slightly different syntax when it comes to CREATE TABLE, CREATE INDEX, and ALTER TABLE statements.
  • Because of this, Micro Focus recommends the following before you attempt to execute the statements:
    • Review the syntax of the SQL before you execute it to ensure it works with your distributed data store.
    • Before executing the SQL, make sure you have userid credentials sufficient to execute DDL statements in your distributed data store.

If you successfully execute the contents of these three files in your target distributed data store before using the Extension Technology, the Extension Technology will mask or subset data accordingly.

Parent topic: Using Distributed Exporter


Using Distributed Sampler

The Distributed Sampler is accessible from Data Builder from [image].

For distributed data stores, the Distributed Sampler is a utility that lets you perform sampling on your data stores and then load that information in the Knowledge Base as part of the Data Inventory process.

Sampling lets you obtain information about the distribution of your data. You can use the distribution as a way to assign classes to other data elements with similar data distributions.

There are three types of sampling:

  • Compressed sampling - Produces the data element fingerprint. The fingerprint graphically shows the distribution of values within a given range for the sampled data element. The fingerprints for numeric and alphanumeric data elements differ in that the fingerprint for an alphanumeric data element shows the distribution of values based on the first character and provides additional information.
  • Standard sampling - Displays information for each data element value including the number of times each value occurred and the percentage that value represents in the total population.
  • Min/Max calculation - Displays the minimum and maximum values for the data element.

Known Restriction: Data Express does not support sampling for binary data.

For more information about the Sampling Options feature, review the Simple Sampling Tutorial in this guide.

  • Sampling Options
  • Validation
    Describes the need for distributed sampling validation based on the type of distributed sampling execution.
  • Load results from file
    During sampling execution, the Extension Technology uses the output files sampling.dat and method.rc to produce the sampling.log file that contains the resulting statistics.
Parent topic: Using Distributed Data Stores with Data Express


Sampling Options

Basic Options

Use the Sampling Options tab in the Distributed Sampler to specify your sampling criteria.

To export your sampling configuration, you need to provide the following information:

  • your workspace (machine ID, company)
  • your source data store credentials (source database, source schema, source user ID/password)
    Note: The source database is an ODBC DSN or Oracle TNS alias configured to access that data store.
  • types of sampling to be performed
  • directory where you want to place output files for Extension Technology (Output directory)
    Note: Only specify an Output directory if you are exporting your sampling configuration.

To export your configuration, click Export. After you have successfully exported from the Distributed Sampler, the following files are created in the specified output directory:

  • method.rc - Encoded connection criteria for each data store referenced.
    Note: If you have performed export functions for masking or subset extraction, the method.rc file already exists in your config directory. If you export your sampling configuration to the config directory, the previous method.rc file will be overwritten.
  • sampling.dat - Information regarding the data elements to be sampled.

Note: If your Knowledge Base and the data stores you want to sample are located on the same Windows machine that is running Data Express, you do not need to export configuration information, execute sampling from a script, or load results. Instead, you would bypass those steps and click the Start button in the Distributed Sampler to execute sampling.

Advanced Options

To further control your sampling results, you can specify advanced sampling options, which are accessible from the Advanced button.

Additional Data Element Attributes

You can restrict which data elements are sampled by their data element types:

Advanced Options - Data Element Types

By default, all types are checked. Uncheck a type to remove it as a sampling candidate.

Note: The Binary data element type represents the COBOL binary (COMP field) data type. True binary is not supported.

Data Element Sizes

You can restrict which data elements are sampled by their data element sizes:

Advanced Options - Data Element Sizes

By default, all values are 0 (zero), which means to include all data elements regardless of the associated size values.

If a minimum value is set, the data element size to be included is the value greater than the specified MinValue. For example, if the Min length is changed to 1, data elements with a length of 2 or greater are included.

Likewise, if a maximum value is set, the data element size to be included is the value less than the specified MaxValue. For example, if the Max length is changed to 99, data elements with a length of 98 or less are included.

Max. and Min. Recalculation Additional Options

You can restrict the Min/Max calculation sampling for data elements:

Advanced Options - Max. And Min. Recalculation Additional Options

The following options are available:

  • Ignore Special Values Zero / Space - If this box is checked, the 0 (zero) and special characters like the space character are ignored in your minimum and maximum value calculations.
  • Out-of-range minimum value - If a defined range within your data element values is desired, the Out-of-range minimum value field defines the smallest value outside of that range. For example, if 5 is specified, values of 6 and greater are included in the range.

    By default, the Out-of-range minimum value is 0 (zero), which means to include all data elements regardless of the actual value.

  • Out-of-range maximum value - If a defined range within your data element values is desired, the Out-of-range maximum value field defines the largest value outside of that range.

    By default, the Out-of-range maximum value is 0 (zero), which means to include all data elements regardless of the actual value.

  • Use range in Min/Max calculation - If this box is checked, the range indicated by the fields Out-of-range minimum value and Out-of-range maximum value is applied to the Min/Max calculation sampling result.

Known Restriction: The Max. and Min. Recalculation Additional Options currently affect the minimum and maximum data element values reported for standard and compressed sampling.

Additional Restriction Options

You can restrict which data elements are sampled based on data element class assignments. You can also reduce the number of data elements to sample from.

Advanced Options - Additional Restriction Options

You can add a class to the table show on the left after selecting it in the Select a Class list. Only data elements that are assigned to a class specified in the Selected Class list will be sampled. This is especially helpful when you have several data stores containing data elements with multiple class assignments.

The Number of records is the total number of records to be sampled. The default value is 0, which means to include all records.

Use the Select distinct values check box to improve the sampling process for big tables. The SELECT DISTINCT command is used to provide the distinct values of a column. This option allows you to create a sampling result only for the distinct values of a column. When checked, already sampled values are discarded and the result will be set for unique ones.

Note: The list of data elements is sorted either numerically or alphanumerically as based on the data element type. Once the limit of records is reached, the remaining data element records are not sampled.
Parent topic: Using Distributed Sampler


Validation

Distributed sampling allows two types of execution:

Direct
Automatically exports comma-separated side files (sampling.dat and method.rc) in the config subfolder of the Data Express extension installation path. It produces sampling results, and then imports those results into the client knowledge base.

This type of sampling requires no validation.

Manual
You can manually export the comma-separated side files, sample manually from the console, and then load your sampling results into the client knowledge base from the Distributed Sampler dialog box in Data Builder.

In this case, validation is useful to ensure that the data stores in the sampling results file match the exported sampling.dat file. This prevents errors and ensures consistent results.

Validation is performed when you specify a Sampling data file in the Distributed Sampler dialog box. See Load results from file for instructions.

Parent topic: Using Distributed Sampler


Load results from file

During sampling execution, the Extension Technology uses the output files sampling.dat and method.rc to produce the sampling.log file that contains the resulting statistics.

The sampling analysis results contained in the sampling.log file must be loaded into the Knowledge Base from the Load results from file tab in the Distributed Sampler. To do this, click the browse button that corresponds to the Sampling results file field; then browse to and select your sampling.log file:

Distributed Sampler - Load results from file

To validate the results, click the browse button that corresponds to the Sampling data file field; then browse to and select your sampling.dat file (optional).

Click Load to load the results.

Once you have loaded your results, you can view them in the Show Synthetic Data Element Contents window.

Parent topic: Using Distributed Sampler


Using Extension Technology

This section of your Getting Started guide provides an overview of the Extension Technology, and provides detailed information on creating masking routines.

You should review the configuration information and the invoking procedure prior to starting the tutorials in Data Express for Distributed Systems Tutorials.

For information regarding hardware, software, user, and configuration requirements, see the Distributed Systems chapter in your Readme . In particular, see section Run-time Knowledge Base Requirements as you must create a Run-time Knowledge Base in order to mask or subset data.

For information regarding installation procedures, see the Distributed Systems Procedures chapter in your Installation Guide .

For more information pertaining to the execution of the ODBC Extension, follow the exercises in Data Express for Distributed Systems Tutorials of this guide.

  • Extension Technology Configuration
  • Extension Technology Directories
  • Invoking Extension Technology
    The Data Express for Distributed Systems Tutorials demonstrates the default method for executing Extension Technology on a Windows platform using default mode.
  • The dxestart command
  • Execution Modes
    This section describes in greater detail the three Extension Technology Execution modes: default, parallel execution, and unload.
  • Error Diagnosis
    Diagnostic information including error messages generated during the execution of the ODBC Extension or Oracle Extension is written to log files.
  • Masking Routines
    A masking routine is a program that contains the processing and masking logic needed to perform the actual masking on your data.
  • Extension Technology Utilities
    This section describes utilities provided with the ODBC Extension and Oracle Extension.
  • Creating an Encoded Connection File
  • Batch Distributed Exporter
Parent topic: Getting Started with Distributed Data Stores


Extension Technology Configuration

Configuration capabilities for the Extension Technology can be globally implemented or can be controlled at the data store level.

  • Extension Technology Configuration File
    Describes the dxeconfig.cfg file used to configure extension technology.
  • Data Store SQL Scripts
  • DXCOMMON Environment Variable
Parent topic: Using Extension Technology


Extension Technology Configuration File

The dxeconfig.cfg file is used to globally manage your ODBC Extension or Oracle Extension settings. By default, it contains the following settings:

* Data Express Configuration File

LogLevel:1
Always Drop Table:N
CSV Data Store:N
COBOL Masking:N
Validate Target Table:N
Validate Source Schema:N
Enable Logging:N
Hide Progress:N
Hide Log Message:N
Use Extended C Masking:N
Trim Spaces for ODBC Char Columns:N
Commit Size(MBs) [user range = from 1 to 4000, default range = 0]:0
Parse String finding LOW-VALUES:N
Create Table using CHAR clause:N
Execute scripts when the process aborts:N
Execute disable/enable CONSTRAINTS:N
Execute Aborted Tables:N

* These entries are only applicable to the Oracle Extension.

Text Output:N
Data Separator:,
Fields Enclosed By:"
Identify Externally:N
Use Append Condition:N

The following settings can be specified:

LogLevel

Controls the level of detail displayed in the dxe_errorlog.txt file. You can change the LogLevel as needed or when requested to do so by Product Support:

LogLevel | Description
1 | Default log level that provides a summary for each processed data store.
2 | Verbose output used for diagnostic purposes that contains the SQL statements executed against your source and target data stores.
3 | Generates ODBC trace output that is written to odbctrace.txt.

Always Drop Table

Determines whether or not to always execute DROP TABLE, and what, if any, additional action is taken. Possible values are:

N | If a table already exists within the data store, the Extension Technology first executes a DELETE FROM TableName statement before performing INSERT operations. Default.
Y | If a table already exists within the data store, it is removed by the execution of the DROP TABLE TableName statement, which forces the table to be recreated.
K | Keep. When the target table is not already deleted, the existing records in the target are kept, and a new record with the same key as another record in the target is discarded.
L | Delete and keep. When the target table is not already deleted, it is deleted with the same query used to extract the record from source, then existing records in the target are kept, and a new record with the same key as another record in the target is discarded.
O | Overwrite. When the target table is not already deleted, the existing records in the target are kept, and a new record with the same key as another record in the target overwrites that record, in effect deleting the original record.
P | Delete and overwrite. When the target table is not already deleted, it is deleted with the same query used to extract the record from source, the existing records in the target are kept, and a new record with the same key as another record in the target overwrites that record, in effect deleting the original record.
Note: When Validate Target Table is set to Y, Data Express performs validation before processing any tables to ensure that the structures of your source and target data stores are exactly the same. If the structures differ and Always Drop Table is set to N, the method does not execute and an error message displays instructing you to change your Always Drop Table setting in order to proceed with the creation of your test environment.

CSV Data Store

Controls whether the tables referenced by a given method should be stored in text files delimited by commas. To enable, change the CSV Data Store parameter from N to Y.

COBOL Masking

Controls whether you are using COBOL routines to mask your data. Possible values are:

Y | By default, COBOL Masking is set to Y.
N | If you want to use C masking routines, change the COBOL Masking parameter to N.
O When the value is "O" (from Others), the product can use any language satisfying the following requirements:
  • Be built to a callable shared object (UNIX), or DLL (Windows), with the relevant entry points exposed/exported as required.
  • Be callable from C.
  • Have the variables coded precisely as per the COBOL copybook UTEUSFLW.CPY (noting that the content/structure differs slightly from the mainframe in order to more closely mimic C data types), both regarding data type and size.
  • Cope with platform-specific byte ordering of numeric data. The C engine code and COBOL copybook expect all numeric parameters to be passed in machine-specific byte ordering. (For example, Intel platforms such as Linux and Windows use the opposite byte ordering to UNIX platforms such as HP-UX, Solaris and AIX).

Validate Target Table

Controls whether the target tables are validated against the source tables to ensure that the structures are exactly the same. By default, Validate Target Table is set to Y. If you do not want to validate the table structures , change this parameter from Y to N.

Note: You may not want to validate the target table because if the validation fails (perhaps due to differing column numbers) dropping the table and then recreating the table will create the table in a different tablespace. Set the Validate Target Table parameter to N only if you are the target and source tables match.

Validate Source Schema

Controls the usage of source schema. If Y, the target schema name is the same as the source one. If N, the default target schema name is used. By default, Validate Source Schema is set to Y.

Enable Logging

For more information about logging, see Logging Exit Routines in the Data Masking Guide.

Hide Progress

Suppresses visual progress during validation.

Hide Log Message

Controls the display of log messages in the dxe_errorlog.txt file when Enable Logging is used.

Use Extended C Masking

Controls the use of extended C masking. If N, standard masking is used. This the default. If set to Y, extended C masking is used. For information on the differences between standard C masking and extended C masking, see Appendix B of the Data Masking Guide.

Trim Spaces for ODBC Char Columns

Optional. If Y, final spaces in ODBC CHAR columns are replaced with low-values (x00). If N or the Trim Spaces for ODBC Char Columns parameter is omitted, final spaces in ODBC CHAR columns are left unchanged.

Commit Size(MBs) [user range = from 1 to 4000, default range = 0]:0

Optional. Controls the frequency of commits, and can be used to process very large numbers of inserts within the same transaction.
0 | A commit is performed immediately after processing 95 MB; one commit for each 95 MB processed. Default.
n | A numerical range from 1 to 4000 indicating the number of processed megabytes (MBs) after which to perform each commit. Any number over 4000 is interpreted as 4000.

Parse String finding LOW-VALUES

Determines whether or not a column is masked based on its use of LOW-VALUES.

N | When the first character of a column set to be masked is LOW-VALUES, the column is not masked. Default.
Y | When the column contains any characters other than LOW-VALUES, it is masked.

Create Table using CHAR clause

When the DROP TABLES option is used, determines whether to create a table using the CHAR data type for CHAR columns only, or to create it using the CHAR data type for both CHAR and VARCHAR columns.

N | When the DROP TABLES option is used, the CREATE TABLE statement uses the CHAR data type for CHAR columns only. Default.
Y | When the DROP TABLES option is used, the CREATE TABLE statement uses the CHAR data type for VARCHAR columns.

Execute script when the process aborts

Determines whether or not to execute pre and post scripts when the process aborts.

N | Pre and post scripts are not executed when the process aborts. Default.
Y | Pre and post scripts are executed when the process aborts.

Execute disable/enable CONSTRAINTS

Determines whether or not Data Express provides ENABLING/DISABLING SQL commands to preserve existing CONSTRAINTS relationships.

N | No constraints are created. Default.
Y | DISABLE/ENABLE constraints are created, but only when Always Drop Table:N is set.

Execute Aborted Tables

Determines whether or not the engine logs all successfully processed data stores or only the failed data store elaborations from the previous run.

N | The engine works normally, and logs the successfully processed data stores. Default.
Y | The engine processes only the failed data store elaborations from the previous run.

The following settings apply only to the Oracle Extension:

Text Output

By default, Data Express inserts the processed data into the specified target data store. You can configure Data Express to write the output to a text file instead, which can be subsequently loaded into an Oracle database using the Oracle sqlldr routine. To enable this, change the Text Output parameter from N to Y.

Data Separator

Oracle Extension only. By default, the delimiter used to separate Oracle fields is the comma (,). You can override this default by specifying one or more characters.

Fields Enclosed By

Oracle Extension only. By default, strings are enclosed by double quotation marks ("). You can override this default by specifying one or more characters.

If the target table does not exist, the Extension Technology executes a CREATE TABLE statement and then INSERT operations in order to create it. However, if the target table already exists, the way the Extension Technology behaves depends on the settings specified in the dxeconfig.cfg file. For example:

Always Drop Table Validate Target Table Resulting Behavior
Y

n/a

A DROP TABLE statement is executed before the CREATE TABLE statement and INSERT operations.

N Y

If the target table's metadata matches the source table's metadata exactly, the method will be executed. If any mismatch in the metadata is found, the method will not be executed.

For example, if the source table is defined as:

 CREATE TABLE SRC.MYTABLE(C1 INT, C2 CHAR(5), C3 DECIMAL(5,2))

and the target table exists, but is defined as:

CREATE TABLE TGT.MYTABLE(C3 DECIMAL(5,2), C1 INT, C2 CHAR(5))

the method will not be executed.

Note: The

The Always Drop Table parameter set to N and the Validate Target Table parameter set to Y are default settings.

N N

The Extension Technology will not perform any validation. If the target table does not include all columns from the source table (regardless of order), the INSERT operation will fail and the elaboration will terminate abnormally.

For example, if the source table is defined as:

 CREATE TABLE SRC.MYTABLE(C1 INT, C2 CHAR(5), C3 DECIMAL(5,2))

and the target table exists, but is defined as:

 CREATE TABLE TGT.MYTABLE(C3 DECIMAL(5,2), C1 INT, C2 CHAR(5))

the method should execute and complete normally.

If the target table is instead defined with a column C5 instead of C3:

CREATE TABLE TGT.MYTABLE(C5 DECIMAL(5,2), C1 INT, C2 CHAR(5))

the INSERT statement will fail, causing the method to terminate abnormally.

Identify Externally

Oracle Extension only. Identifies connection criteria for connecting to source and target databases.
Y | Data Express uses the information in the configured Wallet rather than the method.rc file to connect to the database, eliminating the need to specify user name and password credentials.
N | Database connection requires that you specify a user name and password.

Using append condition

Optional. Oracle Extension only.
Y | During the execution of the dxestart module, adds the APPEND condition to the INSERT statement and the NOLOGGING condition to the CREATE TABLE statement. Important: For the full benefit, you must also set Always Drop Table to Y.
N, or the Using append condition parameter is omitted | Nothing is added during the execution of the dxestart module.

Commit Size

Optional.
0 | Commit is performed after processing all inserts. Default.
n | A numerical range from 1 to 4000 indicating the number of processed megabytes (MBs) after which to perform a commit. This enables incremental commits and should be used when processing a very large number of inserts.
Parent topic: Extension Technology Configuration


Data Store SQL Scripts

Each data-store-specific SQL script contains the following types of information:

  • Data Express configuration options denoted with the prefix DX:
  • SQL statements
  • Comments denoted with the character prefix --

Filenames

The file names used by Data Express to execute data-store-specific operations are:

  • pre_ConnectionAlias.sql - executes or sets options prior to Data Express processing
  • post_ConnectionAlias.sql - executes or sets options after Data Express processing
    where ConnectionAlias specifies:
    • the DSN (data source name) for the ODBC Extension
    • the Oracle TNS service name for the Oracle Extension

For example, with the ODBC Extension, if a given table is accessed via the MYSOURCE DSN and the target data store for that table is MYTARGET, the configuration file names would be named:

  • pre_MYSOURCE.sql
  • post_MYSOURCE.sql
  • pre_MYTARGET.sql
  • post_MYTARGET.sql

Configuration Options

Controlling configuration at the data store level allows data-store-specific SQL operations to be performed during before or after Data Express processing.

The following pre-processing configuration settings can be specified:

  • IdentifierDelimiter

    Specifies two characters to be used as an opening delimiter and a closing delimiter when the data store type requires that the schema name, table name, and column name identifiers needed to be delimited. For example, if working with Microsoft SQL Server data stores, the value would need to be set to DX:IdentifierDelimiter:[]

  • IdentityInsertOn

    Specifies to maintain the identity values from the source database in the target database when the data store type requires that this functionality be enabled. Enabling this parameter causes a SET IDENTITY_INSERT statement to be executed, which will allow values to be inserted into the identity column of a given table. For example, when working with Microsoft SQL Server, DX:IdentityInsertOn:SET IDENTITY_INSERT ~ ON results in SET IDENTITY_INSERT MYSCHEMA.MYTABLE ON

  • EmptyTableCommand

    By default, if the Always Drop Table parameter is set to N within dxeconfig.cfg and the table being processed already exists within the specified schema, the table contents will always be emptied prior to inserting processed values. (Data Express accomplishes this by executing the DELETE FROM statement, which removes rows from your table one at a time.) You can override this default by choosing a different option that allows your data store to perform more optimally such as the option TRUNCATE TABLE, which results in the execution of the TRUNCATE TABLE statement being executed. For example, DX:EmptyTableCommand:TRUNCATE TABLE results in TRUNCATE TABLE MYSCHEMA.MYTABLE

The following post-processing configuration setting can be specified:

  • IdentityInsertOff

    Specifies the default behavior of using the automatically generated value for the identity column in the target database, which may cause result in differing identity values for the source and target databases. Disabling this parameter causes a SET IDENTITY_INSERT statement to be executed. For example, when working with Microsoft SQL Server, DX:IdentityInsertOn:SET IDENTITY_INSERT ~ OFF results in SET IDENTITY_INSERT MYSCHEMA.MYTABLE OFF

Sample Scripts

For your convenience, some sample scripts containing some of the options described in Configuration Options are provided in the config directory as templates in order to assist you in building your specific configuration:

  • pre_Oracle.sql
  • post_Oracle.sql
  • pre_Microsoft SQL Server.sql
  • post_Microsoft SQL Server.sql
Parent topic: Extension Technology Configuration


DXCOMMON Environment Variable

Applies only when using the Oracle Extension on a Windows platform.

During the extension elaboration process, the Oracle extension creates a temporary text file, dxe_common.txt, to track the execution chain. It also deletes this file at the end of the extension elaboration process. Because dxe_common.txt is created in the log subdirectory of the Data directory, when that directory is on a network share, the process of creating and deleting this file can degrade overall performance. Therefore, if you have changed the default Data directory for extension technology to a shared directory on a network, set the DXCOMMON environment variable to improve performance.

The DXCOMMON environment variable enables you to specify a local directory in which to create the dxe_common.txt file.

To implement this environment variable, on a machine that runs Data Express, create a new system variable, DXCOMMON, and set its value to a local directory. For example:

DXCOMMON=C:\dxecommon

If the specified directory does not exist, the Oracle extension creates it when it creates the dxe_common.txt file for the first time.

Parent topic: Extension Technology Configuration
Related concepts
Extension Technology Directories


Extension Technology Directories

For the Extension Technology, data files and executable files are stored in separate locations. This section describes these directories.

Executables

The following table shows the default directory containing your Extension Technology executable:

OS | ODBC Extension | Oracle Extension
Windows | C:\Program Files\Micro Focus\Data Express 4.0\odbc | C:\Program Files\Micro Focus\Data Express 4.0\oracle
UNIX | /opt/microfocus/dataexpress/odbc | /opt/microfocus/dataexpress/oracle
Note: The default data and executable directory for UNIX happens to be the same.

You can install to a different root directory if desired.

Data

The data directory is used for data store configuration files and log files.

The following table shows the default data directory needed for the ODBC Extension or Oracle Extension depending upon your platform:

OS | ODBC Extension | Oracle Extension
Windows | %ALLUSERSPROFILE%\Micro Focus\Data Express 4.0\odbc | %ALLUSERSPROFILE %\Micro Focus\Data Express 4.0\oracle
UNIX | /opt/microfocus/dataexpress/odbc | /opt/microfocus/dataexpress/oracle

The path specified for the %ALLUSERSPROFILE% environment variable differs depending on your Windows platform. For example:

  • Windows XP - C:\Documents and Settings\All Users\Application Data\Micro Focus\Data Express 4.0
  • Windows Vista - C:\ProgramData\Micro Focus\Data Express 4.0
Note: When using the Oracle Extension on a Windows platform, if you choose to change the default Data directory to a shared directory on a network, this could affect performance. For information on improving performance in this scenario, see DXCOMMON Environment Variable.

config

The files generated by the Distributed Exporter and the Distributed Sampler must also be stored in the config subdirectory of the data directory as this is necessary for Extension Technology execution.

If you are invoking the ODBC Extension or the Oracle Extension from a Windows environment, you can specify the config subdirectory as the output directory for masked, reduced, or sampled data stores. Otherwise, you can move the files into the default directory manually.

log

After the execution of the ODBC Extension or the Oracle Extension, a dxe_errorlog.txt log file is created within the log directory in a subdirectory named: YYYY-MM-DD_hh-mm-ss. For example:

log\2009-03-16_19-10-08\dxe_errorlog.txt

If errors occur during processing, the error log displays automatically.

If the execution of the ODBC Extension or the Oracle Extension includes subset extraction processing, dxe_summarylog.txt and dxe_statistics.txt. The Oracle Extension also creates dxe_common.txt to track the execution chain; however, this file is deleted at the end of the extension elaboration process. Log files are also created in the same subdirectory.

If the execution includes sampling analysis, the sampling.log file is also created in this subdirectory; however, this file is used to load sampling results into Data Express and is not meant to be human-readable.

Parent topic: Using Extension Technology
Related concepts
DXCOMMON Environment Variable


Invoking Extension Technology

The Data Express for Distributed Systems Tutorials demonstrates the default method for executing Extension Technology on a Windows platform using default mode.

This section walks you through configuration settings necessary for the execution of Extension Technology.

To invoke the Extension Technology:

  1. Copy the files generated by the Distributed Exporter from the output directory you specified, and paste them in the config subdirectory of the appropriate default data directory on Windows.
    Note: For a list of default directories, refer back to the section Data of the chapter Extension Technology Directories.
  2. Ensure that your data store environment is configured appropriately. If you are working with ODBC data stores, an ODBC Data Source Names (DSN) should be configured as required for your source and target data stores. If you will be executing the ODBC Extension from a UNIX environment, the ODBC driver manager and driver environment should be configured. Typically, this includes:
    • The ODBCINI environment variable should point to the ODBC configuration file.
    • The setting of the LIBPATH (AIX), SHLIB_PATH (HP/UX), or LD_LIBRARY_PATH (all other flavors of UNIX) environment variable should include the lib folder under your ODBC Driver Manager installation.
    • Implement other settings as defined by your ODBC/ODBC Driver Manager documentation.
      Note: If you will be executing the ODBC Extension from a UNIX environment, make sure that the UNIX ODBC DSN matches the ODBC DSN specified when you did the export on Windows. If you are making use of C or COBOL masking routines, verify that you have a development environment for building masking routines and a run-time environment for executing masking routines.
  3. Verify the Run-time Knowledge Base has been created and is configured with the appropriate privileges. For more information, see the sections Run-time Knowledge Base Requirements and User Credential Requirements of your Readme.
  4. Navigate to the executables directory, which contains the Extension Technology (ODBC Extension or Oracle Extension).
  5. Execute the appropriate command to invoke the Extension Technology:
    Mode Windows UNIX

    Default

            dxestart ./dxestart

    Parallel

            dxestart -d DirectoryName ./dxestart -d DirectoryName

    Unload

            dxeunload ./dxeunload
    Tip: dxeunload has the same syntax and format as dxestart. See The dxestart Utility for more information on using dxestart.

    The Extension Technology processes all the tables previously specified within Data Express, and reports a status for each data store, for example:

     Data Express Distributed Server 4.0.0
c) Micro Focus (IP) Limited 2003-2009
Table xdb.SRC.CUSTOMER: 21 rows [##########] Reduced to 6 rows.
Table xdb.SRC.ORDERS: 5 rows [##########] Reduced to 3 rows.
Table xdb.SRC.ITEMS: 8 rows [##########] Reduced to 5 rows.
Data Express processing completed OK.
  6. Use the Distributed Statistics Loader to populate run-time statistics into Data Subset Extraction. For more information, see section Distributed Statistics Loader of chapter Extension Technology Utilities.
Parent topic: Using Extension Technology


The dxestart command

dxestart includes parameters for passing connection information about the input and output data stores. This is useful if you have strong security requirements and use scripts for batch processing.

Parameters for dxestart are valid for both Oracle and ODBC extensions.

Syntax

dxestart -d <path> -i<input method.rc> -o<output method.rc>
         [-f <from-step> -t <to-step>]

Parameters

-d <path> specifies that Data Express reads the method.rc location specified by <path>. If not specified, Data Express uses the default configuration folder. For example:
dxestart -d "C:\Testlib1"

This example reads the connection information from the method.rc file in C:\Testlib1\config and creates the log and statistic files in C:\Testlib1\log.

-i <input method.rc> reads the input connection information from a different .rc file (named [FileName].rc, where [FileName] is a name you specify for the file). This file is the functional equivalent of the method.rc file. Specify the name and the path to the new .rc file in this parameter.

-o <output method.rc> reads the output connection information from a different .rc file (named [FileName].rc, where [FileName] is a name you specify for the file). This file is the functional equivalent of the method.rc file. Specify the name and the path to the new .rc file in this parameter. For example:
dxestart -d "C:\Testlib1" -i "C:\Testlib2\method1.rc" -o "C:\Testlib3\method2.rc"

This example reads the input connection information from the file method1.rc that resides in C:\Testlib2, the output connection information from the file method2.rc that resides in C:\Testlib3, and the exported configuration information from C:\Testlib1\config. It then creates the log and statistic files in C:\Testlib1\log.

See Creating an Encoded Connection File for instructions on creating the input and output connection file that can be used with the -i and -o parameters.

-f <from-step> -t <to-step> enables you to specify a range of steps to be executed. <from-step> represents the first step of the range, and <to-step> represents the last step of the range.

Parent topic: Using Extension Technology


Execution Modes

This section describes in greater detail the three Extension Technology Execution modes: default, parallel execution, and unload.

Default Mode

Use the default execution mode when the source data store meets all the schema requirements described in the section Schema Requirements of chapter Requirements of the Readme .

Parallel Mode

Use the parallel mode when you want to execute more than one method simultaneously.

Warning: Use of the parallel mode where multiple methods access the same tables within the target data store(s) is not currently supported.

In order to run a parallel execution, the files created after a successful export from Data Masking or Data Subset Extraction must be stored in a different data directory for each method.

  1. Create a new directory for a method.
  2. Within the newly created directory, create the subdirectory config.
  3. Copy the contents of the default config directory into the newly created config directory.

    For more information about the default config directory, see the section Extension Technology Directories in this guide.

  4. Export or manually move the files generated by the Distributed Exporter into the new config directory.
    Note: Bear in mind that the machine where the ODBC Extension is executed may not be the same the same machine where the Windows client software is running.
  5. Use the dxestart command with the -d option to specify the new location.

    See Invoking Extension Technology for more information.

Note: You can execute the first method in default mode, which utilizes your default data directory, but you must execute all subsequent methods in parallel execution mode.

For more information about default directories, see the section Data in the chapter Extension Technology Directories.

Unload Mode

In unload mode, the same schema requirements apply (as described in the section Schema Requirements of chapter Requirements of Readme ) , but with a difference: Data Express does not use privileges against the source data store. Instead, information is unloaded from the source data store to separate staging data store where these privileges are can be authorized.

You can use unload mode for the following two scenarios:

Source Data Store | Staging Data Store | Target Data Store
Data Store A | Data Store B | Data Store C
Data Store A | Data Store C | Data Store C

These examples show that the staging data store may or may not be the actual target data store.

For example, to use unload mode with Data Masking:

  1. In the Distributed Loader within Data Builder, specify your source data store as the Source database (as normal), and upload tables.

    For more information about the Distributed Loader, see the section Using Distributed Loader of Part 2. Using Distributed Data Stores with Data Express.

  2. In Data Masking, enable masking of all the tables catalogued.

    For more information, see Enable Masking within the section Data Masking of Part 4: Data Express for Distributed Data Stores Tutorials.

  3. In the Distributed Exporter within Data Masking, specify your staging data store as your Target database, and then export files.
  4. Copy the exported files into the config directory.
    Note: For more information about default directories, see the section Data in the chapter Extension Technology Directories.
  5. Execute the Extension Technology using unload mode.

    For more information, see the section Invoking Extension Technology.

    The data from the source data store is extracted and inserted into the staging data store after the required tables were created.

  6. From this point forward, you can process your data normally provided that you specify the staging data store as your source and the target data store as your target. This means that you load from staging and export into target.
Note: A similar approach can be used to unload with Data Subset Extraction; after using the Distributed Loader to upload your tables, you can mark all the tables as "register tables". Then, it is very simple to create a subsetting method of your register tables from Data Subset Extraction. Use the Distributed Exporter to copy the exported files into the config directory, and execute the Extension Technology using unload mode.
Parent topic: Using Extension Technology


Error Diagnosis

Diagnostic information including error messages generated during the execution of the ODBC Extension or Oracle Extension is written to log files.

These messages are used by administrators and Micro Focus Product Support representatives.

Extension Technology Log Files

The Data Express Extension Technology generates three log files: dxe_errorlog.txt, dxe_summarylog.txt, and dxe_statistics.txt, which are stored in a subdirectory of the log directory. For details, see Extension Technology Directories.

dxe_errorlog.txt

The dxe_errorlog.txt file contains some processing information as well as information on any errors that have occurred during the processing. The level of detail displayed in this log is controlled by the LogLevel configuration option.

A sample of the dxe_errorlog.txt file with LogLevel set to 1 is shown:

 Initialization: populating Data Express Knowledge Base
 -----------------------------------------------------------------------------
Processing table GSDEMO.SRC.CSTOMER
SQLCODE=-28,SQLSTATE=S0002:[XDB][Micro Focus XDB ODBC 32-Bit Driver][GSDEMO]X28
Cannot find table/view/synonym 'SRC.CSTOMER'.
DE-0074: Unable to SELECT from table in the source database.
Ensure that the database permissions provide the 'install' user with SELECT
 privileges.
 Summary: TableID=2 TableName=GSDEMO.SRC.CSTOMER TotalCount=0 ReducedTo=0 CpuTime=
170340.391000 ElapTime=0.000000
 -----------------------------------------------------------------------------
Processing table GSDEMO.SRC.EMPLOYEE
Summary: TableID=4 TableName=GSDEMO.SRC.EMPLOYEE TotalCount=10 ReducedTo=10
CpuTime=1.153000 ElapTime=0.000000

A sample of the dxe_errorlog.txt file with LogLevel set to 3 is shown:

 Initialization: populating Data Express Knowledge Base
 -------------------------------------------------------------------
Processing table GSDEMO.SRC.CSTOMER
dxestart: executing dxeinit GSDEMO.SRC.CSTOMER | dxeextracter GSDEMO.
CSTOMER 1 1 28216 "DATAEXPRSS" 1 1 1 | UDCZIP5 GSDEMO.SRC.CSTOMER |
dxewriter GSDEMO.SRC.CSTOMER 'N' 1 | dxefinal 0 1 2 "DATAEXPRSS"
"DataExpress_Business_Protection" '0' 'N' 1
SQLCODE=-28,SQLSTATE=S0002:[XDB][Micro Focus XDB ODBC 32-Bit Driver]
[GSDEMO]X28 Cannot find table/view/synonym 'SRC.CSTOMER'.
DE-0074: Unable to SELECT from table in the source database.
Ensure that the database permissions provide the 'install' user with
SELECT privileges.
Summary: TableID=2 TableName=GSDEMO.SRC.CSTOMER TotalCount=0
ReducedTo=0 CpuTime=170340.406000 ElapTime=0.000000
 -------------------------------------------------------------------
Processing table GSDEMO.SRC.EMPLOYEE
dxestart: executing dxeinit GSDEMO.SRC.EMPLOYEE | dxeextracter
GSDEMO.SRC.EMPLOYEE 1 1 28216 "DATAEXPRSS" 1 1 1 | UDCZIP5
GSDEMO.SRC.EMPLOYEE | dxewriter GSDEMO.SRC.EMPLOYEE 'N' 1 |
dxefinal 1 1 4 "DATAEXPRSS" "DataExpress_Business_Protection"
 '0' 'N' 1
Extracter: executing SELECT DSE_TABLE.* FROM SRC.EMPLOYEE DSE_TABLE
Writer: executing DROP TABLE TGT.EMPLOYEE
Writer: executing CREATE TABLE TGT.EMPLOYEE (E_NO SMALLINT
NOT NULL, LNAME CHAR(10) NOT NULL, FNAME CHAR(10) NOT NULL, STREET
CHAR(20), CITY CHAR(15), ST CHAR(2), ZIP CHAR(5), DEPT CHAR(4),
PAYRATE DECIMAL(15, 2), COM DECIMAL(2, 2))
Writer: executing INSERT INTO TGT.EMPLOYEE (E_NO, LNAME, FNAME,
STREET, CITY, ST, ZIP, DEPT, PAYRATE, COM) VALUES (?, ?, ?, ?, ?, ?,
?, ?, ?, ?)
Summary: TableID=4 TableName=GSDEMO.SRC.EMPLOYEE TotalCount=10
ReducedTo=10 CpuTime=1.138000 ElapTime=0.000000
Note: Following an abnormal termination, Data Express prompts you to view the dxe_errorlog.txt file. Answer Yes at the prompt to view the file in Notepad (Windows) or cat file |more (UNIX).

The dxe_summarylog.txt file provides a brief summary of the data stores processed during subset extraction with statistics on the number processed and processing time.

A sample of the dxe_summarylog.txt file is shown:

 Machine ID:3 Method:SIMPLE TestRun:'N' Date:20090302
Description:SIMPLE
 STEP   TABLEID   TABLE NAME          ROWS      CPU TIME  ELAPSED TIME
                                ORIGINAL   OUTPUT
 --------------------------------------------------------------------------
 10      2         xdb.SRC.CUSTOMER    21   6    1.23      1.00
 20      6         xdb.SRC.ORDERS       5   3    0.80      0.00
 30      5         xdb.SRC.ITEMS        8   5    0.75      1.00

The dxe_statistics.txt file provides a brief summary of the data stores processed during subset extraction similar to the dxe_summarylog.txt. However, the dxe_statistics.txt file is used by the Distributed Statistics Loader to populate run-time statistics into Data Subset Extraction and is not intended for viewing even though it is human-readable.

Parent topic: Using Extension Technology


Masking Routines

A masking routine is a program that contains the processing and masking logic needed to perform the actual masking on your data.

The masking routines for the Extension Technology can be written in C or COBOL or any other type that uses the same interface and structure as the COBOL exits.

Note: This section is only intended to provide additional information regarding Extension Technology masking routines for the Micro Focus Data Express for Distributed Systems solution. Use the Data Masking Guide as your definitive Data Masking resource.

The ODBC Extension ships with pre-built masking routines, as documented in the Data Masking Guide .

Using Predefined Masking Routines

A number of masking routines are provided with your Data Express installation. For your convenience, these routines are built and ready to use. The source files are provided in the c and cobol subdirectories of src within your Data Express installation directory, along with template sources to assist in the creation of new routines.

C Masking Routines that Access Replacement Lookup Data

The following C routines require access to a data store that contains replacement lookup data:

  • UDCADDC
  • UDCCDFC
  • UDCCFPI
  • UDCCOMC
  • UDCNAMC
  • UDCSNC2
  • UDCSURC
Note: To use C masking routines that access replacement lookup data:
  1. Create and populate the lookup tables.
    Data Express includes the SQL script kb.sql that can be used to create and populate the lookup tables with predefined lookup data that works with the routines listed previously. The script may need to be amended to suit the requirements of your data store. For example:
    • The size of the replacement lookup data needs to be sized appropriately so that the masked value will fit into your user table. For example, if the column being masked is defined as a CHAR (30), you must ensure that the replacement lookup data is no larger than 30 characters in length.
    • Your data store may not require semi-colon terminators for SQL statements
    • If the data store housing your lookup data is an Oracle database, you will need to change the data type VARCHAR to VARCHAR2.
  2. Configure Data Express so that it can access the data store housing your lookup data by editing hdblayer.ini, which is located within the executables directory of your installation.
    Note: No hdblayer.ini configuration is needed if you create lookup tables in the same place where you create the runtime knowledge base. Edit information in the file as follows:
    RDBMS = ConnectionType
 DBNAME = ConnectionAlias
    DBOWNER = TableOwner
    USER = UserID
 PSWD = Password
    where:
    • ConnectionType: Set to ODBC for the ODBC Extension or ORACLE for the Oracle Extension.
    • ConnectionAlias: Set to the data store containing the lookup data.

      For the ODBC Extension, this is the data source name (DSN).

      For the Oracle Extension, this is the Oracle TNS service name as specified in your tnsnames.ora configuration file.

    • TableOwner - The table qualifier for the lookup tables.
    • UserID & Password - The connection criteria for the data store containing the lookup tables.

      For example:

       RDBMS = ODBC
DBNAME = MYDSN
DBOWNER = MYOWNER
USER = MYUSER
PSWD = MYPASSWORD
  3. Encrypt the hdblayer.ini file by executing the appropriate command.
    OS Command

    Windows

            dxeencrypt hdblayer

    UNIX

    ./ dxeencrypt hdblayer

  • C
  • COBOL
Parent topic: Using Extension Technology
Related concepts
Appendix B: C Masking Routines for ODBC - Technical Description
Modification Exit Routines


C

The source files are provided in the c and cobol subdirectories of src within your Data Express installation directory, along with template sources to assist in the creation of new routines.

Creating C Masking Routines

To create a C masking routine:

  1. Make a copy of the template masking routine UDCDEMO.c file.
  2. Update the CHANGER_NAME definition by replacing the name CHANGER with the name of your masking routine:
     #define CHANGER_NAME "CHANGER" /* Apply changer's name here! */
  3. Review the processing logic in the ProcessColumns() function in order to verify that the data type is correct and that the masking routine specified for that data element is in this particular routine:
     /* Data processing example */
if ((pData->nType[i] == TYPE_STRING ||
pData->nType[i] == TYPE_INT_STR ||
pData->nType[i] == TYPE_CLOB) &&
strcmp(pTable[i].szChangerProgram, CHANGER_NAME) == 0)
{
pData->uData[i].szData = ProcessString(pData->uData[i].szData, &pTable[i])
;
}
  4. Code your masking logic within the ProcessString() function.
  5. Edit the makefile:
    • Add an entry to the CHANGERS macro for your new masking routine.
    • Apply the appropriate rules for building your masking routine:

    For your Windows makefile, copy the entry below and then replace UDCDEMO with the name of your masking routine:

     UDCDEMO.obj: dxe_common.h $(COMMON_HEADERS) UDCDEMO.c
    !$(CPP) $(CFLAGS) $*.c

    For your UNIX makefile, copy the entry below and then replace UDCDEMO with the name of your masking routine:

     UDCDEMO: libdecommon.a UDCDEMO.o
    $(CC) $@.o $(LDFLAGS) -o $@

Building C Masking Routines

On Windows platforms, build your C masking routine by invoking nmake from a Visual Studio command prompt. On UNIX platforms, build your C masking routine by invoking make.

Parent topic: Masking Routines


COBOL

In order to work with COBOL masking routines, you need to have the COBOL environment configured:

  • For building the routines, you need the COBOL development environment.
  • For executing the routines, you need the COBOL run-time environment only.

Building COBOL Masking Routines

To create a COBOL masking routine:

  1. Make a copy of the template masking routine UDCDEMO.CBL.
  2. Update the PROGRAM-ID clause to include the name of your masking routine.
  3. Insert your masking logic within the PROCEDURE DIVISION, after the following comment:
     *
  * Insert processing logic here.
     *
  4. Update the build script to add instructions for building the new module. For Windows platforms:
    Step Action Details

    1

    Update the build.bat script.

    Add the appropriate COBOL command for generating the source to object code (.obj), for example:

    cobol

                          MYMODULE
.CBL DEFAULTBYTE"00" CASE:

    Note that routines should be compiled with the DEFAULTBYTE "00" CASE compiler directives.

    Update the setting of the CHANGERS variable to add your masking routine name.

    2

    Update DE_CHANGERS.DEF.

    Add an entry for your masking routine at the end, using a unique value, for example:

    MYMODULE @101

For UNIX platforms, update the build script by adding the appropriate COBOL command for generating the source to object code (.o), for example: cob -xc MYMODULE.CBL -C "DEFAULTBYTE(00) CASE"

Building COBOL Masking Routines

On Windows platforms build your COBOL masking routine by invoking build. On UNIX platforms, build your COBOL masking routine by invoking ./build.

Parent topic: Masking Routines


Extension Technology Utilities

This section describes utilities provided with the ODBC Extension and Oracle Extension.

To use the utilities, you need the connection alias, username, and password for your data store.

dxegenloadfile

If your Windows machine does not have direct access to your data store, data can be loaded into the Data Express Knowledge Base from files. The dxegenloadfile utility works with the Distributed Loader to load table metadata and referential integrity information from files into the Knowledge Base.

The syntax is:

dxegenloadfile -c - d<path> -i<input method.rc>

where:

-c specifies that there is no console interaction. The information about the connection comes from the standard method.rc file. It contains the connection information including database name.

If -c is the only parameter specified, Data Express reads the method.rc file from the default configuration folders. The dxegenloadfile utility generates files which are written to the default configuration folders.

-d <path> specifies Data Express reads the method.rc file from the folder indicated by <path>. For example:

dxegenloadfile -d "C:\Testlib1\config"

This example reads the connection information from the method.rc file in C:\Testlib1\config and creates the list and log file in C:\Testlib1\log.

If not specified, Data Express uses default configuration directories.

Note: The -d and -c parameters are mutually exclusive. If -d is used, then -c must not be specified.

-i <input method.rc> reads the input connection information from a different .rc file (named [FileName].rc, where [FileName] is a name you specify for the file). This file is the functional equivalent of the method.rc file. Specify the name and the path to the new .rc file in this parameter. For example:

dxegenloadfile -d "C:\Testlib1" -i "C:\Testlib2\method1.rc"

This example reads the connection information from the file method1.rc that resides in the C:\Testlib2 folder creates the file list and log file in the folder C:\Testlib1\log.

See Creating an Encoded Connection File for instructions on creating the input connection file that can be used with the -i parameter.

To load data from files, you must first install the ODBC Extension or Oracle Extension and then run the dxegenloadfile utility to generate the text files DataStoreLoader.txt and RILoader.txt, which are placed into the log folder.

For more information about the Distributed Loader, see section Load from File in Part 2. Using Distributed Data Stores in this guide.

dxelc

The dxelc utility is designed to update the knowledge base, changing data structures from a previous release. The input of this command is the output of dxegenloadfile.

The syntax is:

dxelc -d "<lcmap.rc>"

It uses a configuration file that includes the following information in the order shown:

  1. Full path to the output generated by the dxegenloadfile command
  2. Machine ID
  3. Company ID
  4. ODBC instance name for reference environment
  5. Schema name for reference environment
  6. ODBC instance name for Data Express
  7. Schema name for Data Express

Example:

 <full_path_DataStoreLoader.txt>,<machine_id.>,
   <company_id.>,<reference_ODBC>,<reference_schema>,
   <kb_ODBC>,<kb_schema>
Note: To get lcmap.rc, encrypt lcmap.ini using dxeencrypt lcmap.ini.

dxegentypeinfo

dxegentypeinfo is an ODBC Extension utility used by Micro Focus for diagnostic purposes. The utility generates a file dxe_odbcdiag.txt that lists details of ODBC data types supported by the data store, which is placed into the log folder.

Distributed Statistics Loader

For the Data Express for Distributed Systems solution, run-time statistics recorded during subset extraction execution are not automatically loaded into your Data Express Knowledge Base (KB).

The Distributed Statistics Loader is a utility that lets you load run-time statistics into the KB. Once you have executed the Extension Technology to subset your data, a log filed called dxe_statistics.txt is created. This file is used to load the run-time data into the KB so that the statistics are available in Data Subset Extraction.

Use the Distributed Statistics Loader after you have executed the Extension Technology to subset your data.

To use the Distributed Statistics Loader:

  1. Use FTP, NFS, or some means to copy the dxe_statistics.txt file generated by the Extension Technology to the client machine where you have the Data Express Window client-side software installed.
  2. Start Data Subset Extraction.

    See Starting Data Subset Extraction in your Data Subset Extraction Guide for specific instructions.

  3. Click [image] (Distributed Statistics Loader).
  4. In the Log file to import box, specify the dxe_statistcs.txt log file name including the path where you saved it on your Windows machine.
  5. Click Check Log File to verify that the log file you select is correct.

    The remaining fields in the Distributed Statistic Loader windows populate from information specified from the dxe_statistics.txt file.

  6. Click Load to load the subset extraction statistics into Data Subset Extraction.

    A Statistic loaded successfully! message is displayed.

    You can view the run-time statistics in various places in Data Subset Extraction. For example:

  • Viewing the List of Elaborations in Step by selecting the appropriate method from the Main Window
  • Viewing the Data Stores Elaboration from the Elaborations tab in the Work with Method window
  • Viewing the List of Elaborations in Step from the Method Overview tab in the Work with Method window

Run-time statistics include the following:

  • Number of Read Records
  • Number of Written Records
  • Reduction in Percentage
  • Simulation Date
  • Real Extraction Date
  • Elapsed Time [hh:mm:ss]
  • CPU Time [hh:mm:ss]
    Note: The Reduction in Bytes run-time statistic is not used in the Distributed Systems solution because of the way ODBC-enabled data stores display the record length.
Parent topic: Using Extension Technology


Creating an Encoded Connection File

You can create an encoded connection file by following these instructions:

  1. Create a Filename.ini text file that contains at least one line using one of these format specifications:
    dxegenloadfile (-i option) or dxeexport
    Format specification is TYPE,ALIAS,SCHEMA,USERID,PASSWORD where:
    • TYPE is ORACLE or ODBC
    • ALIAS is either your ODBC DSN or Oracle TNS alias
    • SCHEMA is your schema name that contains your tables
    • USERID and PASSWORD are the user credentials to connect to ALIAS
    dxegenloadfile example - ODBC,TUTORIAL,SRC,INSTALL,1
    This example uses ODBC extension to connect to DSN TUTORIAL with supplied user credentials, and retrieves table information from the schema SRC, so dxegenloadfile can create files for consumption by the Distributed Loader.
    dxeexport example - ODBC,Dist KB for DX 4.0,DX40,install,1
    This example uses ODBC extension to connect to DSN Dist KB for DX 4.0 with supplied user credentials, and retrieve knowledgebase information, allowing dxeexport to create required files for dxestart to mask and subset your data.
    dxestart (-i option or -o option)
    Format specification is TYPE,ALIAS,SCHEMA,USERID,PASSWORD where:
    • TYPE is ORACLE or ODBC
    • ALIAS is either your ODBC DSN or Oracle TNS alias
    • SCHEMA is your schema name that contains your tables
    • USERID and PASSWORD are the user credentials to connect to ALIAS
    dxestart -i option example - ODBC,TUTORIAL,SRC,INSTALL,1
    This example uses ODBC extension to connect to DSN TUTORIAL with supplied user credentials, and is the source data store (input). The source data store is be masked, subsetted, and placed in the target data store.
    dxestart -o option example - ODBC,Dist KB for DX 4.0,DX40,install,1
    This example uses ODBC extension to connect to DSN Dist KB for DX 4.0 with supplied user credentials, and is the target data store (output). The source data store is masked, subsetted, and placed in the target data store.
    Note: For any of these formats, you can optionally add ,,,,, to the end of the specification.
  2. Encode a Filename.ini file with dxeencrypt to create an encoded Filename.rc file. For example, from a command prompt, enter:

    dxeencrypt Filename.ini

  3. Use an encoded .rc file with dxegenloadfile, dxeexport, or dxestart.
Parent topic: Using Extension Technology


Batch Distributed Exporter

In addition to invoking the Distributed Exporter directly from the Data Masking or Data Subset Extraction modules themselves, Data Express also provides a batch, command-line utility, dxeexport.

dxeexport

The dxeexport command exports the information required so that a successful masking/subsetting execution can occur when the dxestart command is executed.

Syntax
dxeexport [path/]exportparms [path/]connfile
Parameters
Parameters for dxeeport are valid for both Oracle and ODBC extensions.
path
Full path to the specified file; required when the file is not in the current directory.
exportparms
The name of the file that contains export parameters. This must be a text (.txt) file. It identifies the group and methods to export from the knowledge base. The text in this file consists of the following items, each delimited by a comma:
  • Machine ID
  • Company
  • Group name
  • Method name
  • Step range
  • Test Run(Y,N)
  • Configuration file path (location of your configuration directory)
Note: *ALL is allowed at both the group and at the method level. When *ALL is used, subfolders for each method will be created in the destination directory.
connfile
The name of the file that contains encoded connections. This must be an .rc file. dxeexport uses this file to retrieve required information from the Data Express knowledge base. See Creating a Encoded Connection File for details.
Example
From command line, enter:

dxeexport "C:\AA2\Export_Folder\exportparms" "C:\AA2\Export_Folder\connfile"

This command executes a batch export for the method described in exportparms.txt, using the Data Express knowledge base described in connfile.rc.

Assume that exportparms.txt contains one line as follows:

PRUEBA,PRUEBA,PG1,PM1,0010, 0100,N,C:\AA2\Export_Folder

With this example, the method PM1 of group PG1 on machine PRUEBA and company PRUEBA is exported, from step 10 to step 100, with test run set to No (so real extraction), and the resulting configuration files generated by the process are placed in folder config under C:\AA2\Export_Folder.

You can now run dxestart to mask and subset your data, specifying C:\AA2\Export_folder as the -d argument.

Parent topic: Using Extension Technology


Data Express for Distributed Data Stores Tutorials

These tutorials introduce the Data Express Windows client and show you how to configure your working environment and start and use the three Data Express modules: Data Builder, Data Masking, and Data Subset Extraction.

Follow the tutorials in this guide in order after you have completed the product installation.

  • Using Data Express
  • Data Inventory
    This tutorial describes the Data Inventory process, which includes preparing a work environment using Data Builder, and Definition and association of classes.
  • Data Masking
    Data Masking is used to manage the privacy of data contained in databases of applications that are either developed internally or purchased by external suppliers. The aim is to protect sensitive data such as name, surname, or company name according to privacy laws.
  • Simple Data Subset Extraction
    Data Subset Extraction is used to generate a reduced test environment.
  • Data Subset Extraction using Referential Integrity
    In this tutorial, you import class information using referential integrity.
  • Data Subset Extraction Using a Combined Data Element
  • Simple Sampling
    In this tutorial, you perform sampling to obtain information about the distribution of data. Then you use that distribution as a way to assign classes to other data elements with similar data distributions.
  • Data Generation
    The Data Generation module is used to generate a test environment.
Parent topic: Getting Started with Distributed Data Stores


Using Data Express

The easiest way to learn about Data Express is to complete a few simple tasks. This guide takes you through configuring Data Builder, loading metadata into the Data Express Knowledge Base, using Data Masking to implement masking rules on a small amount of data, using Data Subset Extraction to create a simple subset of data, and finally using Data Subset Extraction twice more to create subsets of data using referential integrity. Please note that some of the subsetting examples also mask data, showing that Data Express can do both functions in one method invocation.

The three Data Express modules (Data Builder, Data Masking, and Data Subset Extraction), and the ODBC Extension contain all the tools you need to perform these tasks.

Machine IDs and Company Codes

A Machine ID and Company code pair is essentially an identifier for your Data Express working environment or workspace. The Machine ID is the higher-level identifier, while the Company code must be identified by a Machine ID. You can create a new Machine ID and a new Company code for each test environment, or you can create new Company codes under a single Machine ID as needed for each test environment you want to create.

Note: A single machine ID can correspond to numerous company codes. A new company code can be created only if an existing machine ID is first selected.

Sample Data

Data Express includes sample XDB tables for the Data Express Knowledge Base and a corresponding XDB source database called TUTORIAL , which is used throughout the exercises in this tutorial. The database and its sample tables are installed automatically during product installation.

  • Preparation
  • Sample Session
Parent topic: Data Express for Distributed Data Stores Tutorials


Preparation

In this session, you configure connectivity for the sample XDB data store provided in the product installation and perform setup for basic Data Express options. This involves setting up a data source name (DSN) to enable communication to the XDB TUTORIAL data store for all three client-side modules.

Note: Sessions in this tutorial should be done in order.
Parent topic: Using Data Express


Sample Session

In this session, you:

  • Configure an ODBC DSN
  • Start Data Builder
  • Set options

Configuring an ODBC DSN

Data Express requires a user or system DSN for connectivity to your data store through an ODBC driver, which is configured using the Data Sources (ODBC) administrative tool.

This section provides instructions for configuring a user DSN needed for the ODBC driver provided with your Data Express installation.

Note: The Data Express product installation will install the correct networking components and ODBC driver.

The Data Express Knowledge Base also uses a system DSN for connectivity through an ODBC driver; the driver is installed and the preconfigured DSN is implemented during product installation.

A BDE alias is not required for your ODBC-enabled data store.

To configure your ODBC DSN:

  1. Start the ODBC Data Source Administrator. (Use Start > Settings > Control Panel > Administrative Tools > Data Sources (ODBC).)
    Note: On some older versions of Windows, you can access Administrative Tools from the Start menu.
  2. Click the User DSN tab.
  3. Click Add.
  4. In the Create New Data Source window, select the driver Micro Focus XDB for DX 4.0.

    Micro Focus XDB ODBC 32-Bit Driver Setup

  5. Enter the following details for the DSN (leave the remaining fields unchanged):
    Data Source Name | GSDEMO
    Server | DX40
    User ID | install
  6. Click OK. The DSN appears in the User Data Sources list.

See section Run-time Knowledge Base Requirements as you must create a Run-time Knowledge Base in order to mask or subset data.

Starting Data Builder

Data Builder enables functionality for Data Masking and Data Subset Extraction, and for the ODBC and Oracle Extension technologies. With Data Builder, you can take an inventory of data, and classify the data as you see fit. These processes store information about the data in the Data Express Knowledge Base.

Note: You will use the settings shown in this section throughout each subsequent chapter of this Getting Started guide.

Start Data Builder:

  1. Click Start > All Programs > Micro Focus Data Express 4.0 > Data Builder.
  2. Select the XDB Knowledge Base Dist KB for DX 4.0 as the required database with which to launch the connection. By default, the highlighted item in the list of available databases is the last database you connected to using Data Builder. However, for the very first connection, Dist KB for DX 4.0 - DX40 is selected by default.
    Note: By selecting Dist KB for DX 4.0 - DX40, you do not need to choose the database and then the schema DX40.
  3. Click OK.
  4. In the User Name box, type: install.
  5. In the Password box, type: 1.
    Note: This is the number one.

    The[Work with Data Stores] window provides a list of files cataloged by Data Express using different sets of search criteria, including:

  • Machine identifiers
  • Company names
  • Application identifiers
  • Data store types
  • Data stores with assigned classes (Multiple class Selection)
  • Data stores with Data Changer activated
  • Data stores with Data Changer inactivated
  • Data stores set as Register Table
  • Data stores without Register Table attribute

This window comprises four tabular views: Structure, Classes, Data Changer, and Register Tables. You can select a tab to view files using a specific set of criteria as listed previously. Each tab is associated with different grouping criteria.

The list of displayed data stores can be further restricted by applying filters. Filters can be selected from the context menu displayed by right-clicking the List of Files grid, or by selecting the specific icons displayed on the left side of the screen.

Note: Data Masking and the Data Subset Extraction are launched as separate applications.

Setting Options

Let's look at some ways you can configure Data Express behavior.

Default Environment Parameter Settings Selection

You can set default parameters for the environment, identified by the Machine ID and Company pair. The main parameter, Output Date Format, indicates the format in which dates are displayed in Data Express, for example:

  • DD/MM/YY or DD/MM/YYYY - day/month/year (year with 2 or 4 digits)
  • MM/DD/YY or MM/DD/YYYY - month/day/year (year with 2 or 4 digits)
Note: The Output Date Format parameter is valid for the whole environment and is therefore independent of the Company parameter. To define or change this parameter, click Environment > Work With Default Parameters.

Client Options

Options can be edited in the Options window:

  1. To display the window, click Tools > Options.
  2. In the Help language list, select English.
  3. As a security measure to not store password information for the current connection with the Knowledge Base, leave the box Save user name and password unchecked.
  4. Click the Filter Settings tab.

    The Filter Settings tab allows the scope of the filters configured in the Work with Files area to be selected.

  5. Select Filter on all TabSheets so that all filtering properties remain regardless of what tab (Structure, Classes, Data Changer, or Registers Tables) in the Data Builder - Work with data stores window is active.
  6. Leave the box Show filter before loading data unchecked.
  7. Click the Confirmation Messages tab.

    The Confirmation Messages tab lets you set specific work modes.

  8. Leave all the default settings, and click Apply.
  9. Click OK.
  10. If you're planning to go straight on to the next session, you can keep Data Builder open. Otherwise, either Click File > Exit or click the [image] button, and then click Yes.
Parent topic: Using Data Express


Data Inventory

This tutorial describes the Data Inventory process, which includes preparing a work environment using Data Builder, and Definition and association of classes.
Note: Sessions in this tutorial should be done in order.

Work through the exercises in this tutorial only after you have completed all exercises in the previous tutorial.

To begin, you must already be logged in to Data Builder.

  • Sample Session
Parent topic: Data Express for Distributed Data Stores Tutorials


Sample Session

In this session, you:

  • Set your workspace
  • Load data store information into the Knowledge Base

Setting Your Workspace

Data Builder lets you set your workspace as part of the Data Inventory process. You must specify your workspace to organize and sort data on the basis of a logical model:

  1. From Data Builder, open the Work with Machine IDs window (click [image] or access it through Environment > Work with Machine IDs) to create a new Machine ID.
  2. Click New.

  3. Specify your name as the name of the machine ID in the Machine ID box. For this tutorial, type: ALLIE.

  4. Type a description of your machine ID in the Description box. For this tutorial, type: ALLIE'S ENVIRONMENT.
  5. Specify additional information if appropriate. For this tutorial, the additional fields have been left blank. Click Apply.

  6. Click OK to save your specification and to close the window.

  7. From Data Builder, open the Work with Companies window (click [image] or access it through Environment > Work with Companies) to create a new company code:
  8. Click New.
  9. Select the ALLIE Machine ID you just created.
  10. Specify DXPROJ as the name of your company code in the Company Code box.
  11. In the Company Description box, type: DX GETTING STARTED GUIDE.
  12. By default, the Life Cycle is activated. Leave this box selected.
  13. By default, the Is a Backup Company checkbox is not selected. Leave this box unselected.

    During the definition of a company, the backup company associated with it can also be defined.

    A Backup Company is useful when all the information concerning the files that undergo the Life Cycle procedure needs to be preserved.

    Note: Only one version for each file is stored in the backup company.
  14. Click Apply.

    The company you just created appears in the List of Companies.

  15. Click OK to save your specification and to close the window.

  16. Continue to the next section.

Loading your Data Store Information into the Knowledge Base

Perform this procedure to map and load the definitions from your data store into the XDB Knowledge Base. The information from your data store will be mapped to your workspace.

Load data store information into the Knowledge Base:

  1. Open the Distributed Loader window by clicking [image].
  2. Verify that the Machine ID ALLIE and the Company DXPROJ are selected.
    Note: If this is your first time using the Distributed Loader, ALLIE and DXPROJ will be automatically selected.
  3. For this tutorial, leave the load option Skip selected.
  4. In the Source database list, select GSDEMO.
  5. In the Schema box, type: SRC.
  6. In the User name box, type: install.
  7. In the Password box, type: 1.
    Note: This is the number one.
  8. Click Preview Tables to view all the available tables in the source database.
  9. Check the desired tables to load.
    Note: If you would like to limit the tables displayed by schema, in the Schema box, type the full schema name for the tables you want to view, and then click Preview Tables again.

    By default, all tables are selected. You can deselect tables as appropriate.

  10. Click Load to begin the export of the metadata into the Knowledge Base.
  11. Click Yes. An Insert tables window appears, followed by an Information window. After the metadata has fully loaded, an Information window appears that displays data store information including the number of data stores.
  12. Click OK.
  13. Click Yes to display the log file in a text format.
  14. Close the notepad view of the log file.
  15. Click Close to close the Distributed Loader.

    The list of Data Stores in the [Work with Data Stores] window is updated automatically.

  16. 16. If you're planning to go straight on to the next session, you can keep Data Builder open. Otherwise, either Click File > Exit or click the [image] button, and then click Yes.
Parent topic: Data Inventory


Data Masking

Data Masking is used to manage the privacy of data contained in databases of applications that are either developed internally or purchased by external suppliers. The aim is to protect sensitive data such as name, surname, or company name according to privacy laws.

This tutorial explains the Data Masking configuration procedure, and describes the steps required to migrate the files in your application.

Note: Sessions in this tutorial should be done in order.

This tutorial walks you through the data classification and the data masking phases.

Begin this session only after you have set your workspace and loaded data store information into the Knowledge Base in tutorials Using Data Express and Data Inventory. You must already be logged in to Data Builder to begin.

  • Sample Session
Parent topic: Data Express for Distributed Data Stores Tutorials


Sample Session

In this session, you:

  • Create a masking class
  • Associate a masking class to data elements
  • Start the Data Masking module
  • Enable masking
  • Export masking information
  • Execute masking

Creating a Masking Class

You can create new classes to be used for masking purposes, or you can use a predefined class. In this exercise you will create a masking class named ZIPCODE.

  1. In Data Builder, open the Work with Classes window (click [image] or access it through Environment > Work with Classes).
  2. Click New.
  3. In the Name field, specify ZIPCODE as the name of the class you want to create.
  4. In the Full description field, specify ZIPCODE as the description of the class.
    Note: The description will be visible as a label when you assign the class to data elements within data stores.
  5. Specify UDCZIP5 in the Data Masking Routine box.
    Note: This masking routine is provided during product installation.

    You can mask at the class level or at the data-element level. In this tutorial, we will be masking at the class level.

  6. Select the Super Class Privacy.
  7. Select the data type Alpha.
  8. Leave the remainder of the fields at their default values. Click Apply.

    The class ZIPCODE is now listed at the bottom of List of Classes.

  9. Click OK.
  10. Continue to the next section.

Associating a Masking Class to Data Elements

In this exercise you will be manually associating the ZIPCODE class you just created with columns in tables CUSTOMER and EMPLOYEE.

  1. Open the Work with Data Elements window (click [image] or access it through Environment > Work with Data Elements).
    Note: If a message appears indicating that the default confirmation options have not been set, click Yes to set them; click Apply, and then OK in the Options window. However, this should not occur if you followed the previous steps in the tutorial Using Data Express.

    You can also access the Work with Data Elements window from the main window by selecting the file name, right-clicking to view the context menu, and then by clicking Work with Data Elements of the Selected Data Store.

  2. Click the Apply Filter button to display all data stores:
    Note: To narrow down the list of data elements displayed, use the filter fields (Machine ID, Company name, and Application ID), and then click Apply Filter. To display all data elements, simply click Apply Filter.
  3. Click the Classes - Assignment tab.
  4. In the All Classes pane, expand the Privacy Super Class, and then select the class ZIPCODE.
  5. Click the Data Store column to sort data elements by data store.
  6. In the List of Data Elements, select the row for CUSTOMER.ZIP. (That is, select the row for the data storeCUSTOMER and the name ZIP.)
  7. Click the Assign Class button to assign the ZIPCODE class to CUSTOMER.ZIP.

    ZIPCODE now appears in the Assigned Class column for CUSTOMER.ZIP.

  8. In the List of Data Elements, select the row for EMPLOYEE.ZIP. (That is, select the row for the data store EMPLOYEE and the name ZIP.)
  9. Click the Assign Class button to assign the ZIPCODE class to EMPLOYEE.ZIP.

    ZIPCODE now appears in the Assigned Class column for EMPLOYEE.ZIP.

  10. Click the Close button to close the Work with Data Elements window.
  11. If you are planning to go straight on to the next tutorial Simple Data Subset Extraction, you can keep Data Builder open. Otherwise, either Click File > Exit or click the [image] button, and then click Yes.
  12. Continue to the next chapter.

Starting the Data Masking Module

To start Data Masking:

  1. Click Start > All Programs > Micro Focus Data Express 4.0 > Data Masking.
  2. Click OK.
  3. Select the XDB Knowledge Base Dist KB for DX 4.0 as the required database with which to launch the connection. By default, the highlighted item in the list of available databases is the last database you connected to using Data Masking.
  4. Click OK.
  5. In the User Name box, type: install.
  6. In the Password box, type: 1.
    Note: This is the number one.
  7. Click OK.
  8. Select the schema DX40, and click OK.

    The Main Window comprises four main areas: Machine ID./Company grid, All Machine ID./Companies, Multiple Class Selection, and Object Distribution.

    The area containing the grid and the tabs Enabled and Disabled is referred to as the Machine ID./Company area or the Grid area. The grid lists files related to the selected class and to the selected machine ID and company. The names of the forms, their meaning and the included information are outlined below.

    The Enabled tab shows a list of all active files (enabling a file means that it is visible in the export functionality in order to be prepared for masking), and all files, which have one or more specific classes related to their data elements.

    The Object Distribution area displays a pie chart and bar chart representation of the file types for the classes selected in the Machine ID./Companies or Multiple Class Section area. The pie chart shows the percentage distribution of file types, while the bar chart shows the numerical distribution of file types. For both charts, both enabled and disabled files are included. Click the minimize button (_) to minimize the Object Distribution area.

  9. Continue to the next section.

Enabling Masking

Data Masking lets you to enable masking for individual data stores or all data stores, and to enable masking at the data-element level or class level.

  • You can mask at the class level or at the data-element level. In this tutorial, we will be masking at the class level.

In this exercise you will activate masking for all data stores (EMPLOYEE and CUSTOMER) at the class (ZIPCODE) level.

Note: To mask at the data-element level, double-click the desired data store from the Main Window, right-click the desired data element, and then click Data Changer Data Element Properties. Click Change, click Yes to activate the Changer option, and finally click Apply to activate your setting.

Enable masking:

  1. Click Tools>Enable All Data Stores.
    Note: Only the data stores that have assigned classes with masking routines defined will be enabled.
  2. Click Yes to continue.

    The data stores are now listed in the table on the Enabled tab. For a list of data stores that do not have assigned classes, click the Disabled tab.

  3. Continue to the next section.

Exporting Masking Information

In this exercise you will export your masked information so that the Extension Technology can mask your data.

Export masking information:

  1. Click [image] (Distributed Exporter) from the main window of Data Masking. The Distributed Exporter window is shown:
  2. In the Machine ID list, click ALLIE.
    Note: If you have not created any additional Machine IDs, ALLIE will be selected by default.
  3. In the Company list, click DXPROJ.
    Note: If you have not created any additional Company names, DXPROJ will be selected by default.
  4. Click the Select Target button.
    Note: If you have already completed this tutorial once, your target database will already be listed. If this is the case, you just need to select the row for the GSDEMO database from the Target Database List and then click OK. You can then continue to step 9.
  5. Click Add.
  6. Enter the following details for the target (leave the remaining fields unchanged):
    Database | GSDEMO
    Type | ODBC
    User name | install
    Password | 1
    Note: In this tutorial, the source and target databases are the same.
  7. Click OK to close the Add Target window.

    The database you added now appears in the Select Target Database window.

  8. Click OK to close the Select Target Database window.
  9. In the Distributed Exporter window, in the Target schema field, specify TGT.
    Note: You must specify a Target schema name different from the Source schema name if the target database is the same as the source database.
  10. In the Output directory, specify the odbc\config subfolder of your data directory. For more information, see the section Data of chapter Extension Technology Directory of Part 3. Using Extension Technology. For example: c:\Documents and Settings\All Users\Application Data\Micro Focus\Data Express 4.0\odbc\config
    Note: Files needed for the ODBC Extension and Oracle Extension must be placed in the config directory of your default data directory on Windows. In this tutorial you will be invoking the ODBC Extension from a Windows environment; therefore, you will specify this directory as your output directory.
  11. Click Export.
  12. Click Yes.
  13. Click OK.
  14. Click Close.
  15. Exit Data Masking. Click File > Exit or click the [image] button, and then click Yes.
  16. Continue to the next section.

Executing Masking

By default, the Extension Technology will attempt to insert the processed data into the target database. Execute masking with Extension Technology:

  1. From a MS-DOS prompt, change to the executables directory c:\Program Files\Micro Focus\Data Express 4.0\odbc.
  2. 2. Type dxestart The target ODBC data store now contains masked tables TGT.EMPLOYEE and TGT.CUSTOMER.
  3. The execution finished without errors, so you are prompted to view the log file. Type Y to view the file.
  4. Close the text file.
  5. Continue to the next section.

Verifying Results

Verify that the zip codes in the masked tables are in fact different:

  1. Launch the SQL Wizard:
     ..\mfsql\bin\xwiz40n
  2. Notice that the values in the Zip Code column for SRC.EMPLOYEE and SRC.CUSTOMER are different in TGT.EMPLOYEE and TGT.CUSTOMER.
Parent topic: Data Masking


Simple Data Subset Extraction

Data Subset Extraction is used to generate a reduced test environment.

In this tutorial, you perform basic data subset extraction.

Note: Sessions in this tutorial should be done in order.

This tutorial walks you through the data classification and the data subset extraction phases. This session involves creating simple subsetting classes.

Begin this session only after you have set your workspace and loaded data store information into the Knowledge Base in tutorials Using Data Express and Data Inventory. You must already be logged in to Data Builder to begin.

  • Sample Session
Parent topic: Data Express for Distributed Data Stores Tutorials


Sample Session

In this session, you:

  • Create simple subsetting classes
  • Associate simple subsetting classes to data elements
  • Start the Data Subset Extraction module
  • Create a group
  • Create a simple method within that group
  • Export simple subsetting method information
  • Execute simple subsetting method

Creating Simple Subsetting Classes

You can create new classes to be used for subsetting purposes, or you can use a predefined class. In this exercise you will create subsetting classes named DEPTNAM and DEPTNO.

  1. In Data Builder, open the Work with Classes window (click [image] or access it through Environment > Work with Classes).
  2. Click New.
  3. In the Name field, specify DEPTNAM as the name of the class you want to create.
  4. In the Full description field, specify DEPARTMENT NAME as the description of the class.
    Note: The description will be visible as a label when you assign the class to data elements within files.
  5. Select the Super Class None.
  6. Select the data type Alpha.
  7. Leave the remainder of the fields at their default values.
  8. Click Apply. The class DEPTNAM appears at the end of List of Classes.
  9. Click New.
  10. In the Name field, specify DEPTNO as the name of the class you want to create.
  11. In the Full description field, specify DEPARTMENT NUMBER as the description of the class.
  12. Select the Super Class None.
  13. Select the data type Alpha.
  14. Click Apply. The class DEPTNAM appears at the end of List of Classes.
  15. Click OK to close the Work with Classes window.
  16. Continue to the next section.

Associating Simple Subsetting Classes to Data Elements

In this exercise you will be manually associating the classes DEPTNAM and DEPTNO you just created with columns in tables DEPT and EMPLOYEE:

  1. Open the Work with Data Elements window (click [image] or access it through Environment > Work with Data Elements).
    Note: If a message appears indicating that the default confirmation options have not been set, click Yes to set them; click Apply, and then OK in the Options window. However, this should not occur if you followed the steps in the tutorial Using Data Express.

    You can also access the Work with Data Elements window from the main window by selecting the file name, right-clicking to view the context menu, and then by clicking Work with Data Elements of the Selected Data Store.

  2. Click the Apply Filter button to display all data stores:
    Note: To narrow down the list of data elements displayed, use the filter fields (Machine ID, Company name, and Application ID), and then click Apply Filter. To display all data elements, simply click Apply Filter.
  3. Click the Classes - Assignment tab.
  4. In the All Classes pane, expand the None Super Class, and then select the class DEPARTMENT NUMBER.
  5. Click the Data Store column to data elements by data store.
  6. In the List of Data Elements, select the row for DEPT.DEPT. (That is, select the row for the data storeDEPT and the name DEPT.)
    Note: By selecting data elements, you are specifying to mask at the data-element level. If you do not select any data element, you will be masking at the class level.
  7. Click the Assign Class button to assign the DEPTNO class to DEPT.DEPT.

    DEPARTMENT NUMBER now appears in the Assigned Class column for DEPT.DEPT.

  8. In the List of Data Elements, select the row for EMPLOYEE.DEPT. (That is, select the row for the data store EMPLOYEE and the name DEPT.)
  9. Click the Assign Class button to assign the DEPTNO class to EMPLOYEE.DEPT.

    DEPARTMENT NUMBER now appears in the Assigned Class column for EMPLOYEE.DEPT.

  10. In the All Classes pane under the None Super Class, select the class DEPARTMENT NAME.
  11. In the List of Data Elements, select the row for DEPT.DNAME. (That is, select the row for the data storeDEPT and the name DNAME.)
  12. Click the Assign Class button to assign the DEPTNAM class to DEPT.DNAME.

    DEPARTMENT NAME now appears in the Assigned Class column for DEPT.DNAME.

  13. Click the Close button to close the Work with Data Elements window.
  14. If you are planning to go straight on to the next tutorial Data Subset Extraction using Referential Integrity, you can keep Data Builder open. Otherwise, either click File > Exit or click the [image] button, and then click Yes.
  15. Continue to the next section.

Starting the Data Subset Extraction Module

To start Data Subset Extraction:

  1. Click Start > All Programs > Micro Focus Data Express 4.0 > Data Subset Extraction.
    Note: If you are using Data Express on Windows Vista and User Access Control is enabled, you must run Data Subset Extraction as Administrator.
  2. Select the XDB Knowledge Base Dist KB for DX for 4.0 as the required database with which to launch the connection .
  3. Click OK.
  4. In the User Name field, specify install.
  5. In the Password field, specify 1.
    Note: This is a number one.
  6. Click OK.
  7. Select the schema DX40. The Data Subset Extraction window appears:
  8. The Data Subset Extraction window comprises two main areas: the node pane and the grid area. The node pane contains a hierarchical structure that shows the grouping associated with each root-level node. The two root-level nodes are Groups and Creators.
  9. Continue to the next section.

Creating a Group

The Work with Groups window lets you to define, view, modify, and delete groups.

Groups can be used to logically separate the methods created for each workspace. Each method must belong to a group.

To create a group:

  1. Open the Work with Groups window (click [image] or access it through Environment > Work with Groups).
  2. Click New.
  3. In the Machine ID list, click ALLIE.
  4. In the Company list, click DXPROJ.
  5. In the Group name box, specify DX40GS.
  6. In the Group description box, type DX 40 GETTING STARTED GUIDE.
  7. Click Apply.
  8. Click OK.
  9. Continue to the next section.

Creating a Simple Method

The New Method Wizard is used to create a new method, which is the extraction proposal containing the set of operations needed for the creation of the test environment. During method creation, all the information concerning the method is saved locally in a directory indicated by the user

Reduction can be obtained for data stores of a certain size and/or a particular type. In this case, you must be knowledgeable about the application to determine the appropriate selections. However, Data Subset Extraction can recognize and automatically create a first subset containing a series of files (called register table files) that are initially brought into the new test environment. Identifying register table files has the following advantages:

  • You can create a method to extract only the register table file subset, which is normally valid for the creation of any test environment independently of the selection criteria applied. Once this method has been saved and confirmed, it can be run whenever necessary.

You can create a second file subset, which filters the register file subset you created, to produce a file subset that is reduced even further.

To create a simple method:

  1. Launch the New Method Wizard (click [image] or access it through Environment > Create New Method).
  2. In the Machine ID list, click ALLIE.
  3. In the Company list, click DXPROJ.
  4. Note: Files belonging to different workspaces cannot be included.
  5. Click Next to go to the next screen. Note: If a group has not previously been defined in the Work with Groups area, it can be defined from this screen, by simply specifying its name and description in a specific field.
  6. In the Group name box, click DX40GS.
  7. Text in the Group description box populates automatically.
  8. In the Method box, type SIMPLE.
  9. In the Method description box, type: SIMPLE METHOD.
  10. Click Next to go to the next screen.
  11. Select Import all non-register table Data Stores to include demographic files are registered in a dedicated elaboration step of the method.
  12. Click to clear the Import all register table Data Stores box.
  13. Click Next to go the next screen:
  14. Select DEPARTMENT NAME to be used as the primary extraction criteria.
  15. Click Next. Data Subset Extraction automatically extracts all the environment files containing the selected classes, entering them into a specific step. Once the method has been created, the remaining files (that is the files that are not sensitive to the selected classes) will initially be excluded from elaboration. During the Method Confirmation phase, they can be included in the method again. These files will then be integrally copied into the new environment generated by the method involved.
  16. One or more classes related to each class selected as a primary extraction step can be indicated. The primary extraction criteria are listed in the Selection class field.
  17. Select the class DEPARTMENT NUM BER from the Other class list.
  18. Click Add to relate the class DEPARTMENT NUMBER to the class DEPARTMENT NAME. Data Subset Extraction automatically extracts all the environment files containing the related classes and inserts them in a further elaboration step.
  19. Click Finish. The Create Method window is displayed.
  20. Click Start.
  21. Click OK.
  22. Click Confirm. During method confirmation, specified filters are applied to the classes needed to perform the extraction, and the method information is imported from a temporary file to the Knowledge Base. This process, by default, will only include the data stores that contain these selected classes. If you would like to include your other data stores not related to the classes specified previously, check the option Move excluded data stores into new step. In this example, we will simply create a method based on the data stores that use classes DEPTNAM and DEPTNO.
  23. Click OK.
  24. In the Choose a Filter Type list, click FILTER BY VALUE LIST:
  25. In one of the boxes on the lower-left corner, type Sales.
  26. Click OK to close the Set Filter to Selection Classes window.
  27. Click OK to confirm the method.
  28. In the Work with Method window, click Properties.
  29. Click Active.
  30. Click OK to close the Method Properties window.
  31. Click OK to close the Work with Method window.
  32. Continue to the next section.

Exporting Simple Subsetting Method Information

Export simple subsetting method information.

  1. Open the Distributed Exporter window by clicking from the main window:
  2. Click Select Target.
    Note: If you have already completed the Masking tutorial, your target database will already be listed. If this is the case, you just need to select the row for the GSDEMO database from the Target Database List and then click OK; then proceed to step 7.
  3. Click Add.
  4. Enter the following details for the target (leave the remaining fields unchanged):
    Database | GSDEMO
    Type | ODBC
    User name | install
    Password | 1
  5. Click OK.
  6. Click OK.
  7. In the Distributed Exporter window, in the Target schema field, specify TGT.
  8. In the Output directory, specify the odbc\config subfolder of your data directory. For more information, see the section Data of chapter Extension Technology Directory of Part 3. Using Extension Technology.
  9. Click Export.
  10. When prompted to continue elaboration, click Yes.
  11. Click OK.
  12. Click Close to close the Distributed Exporter window.
  13. If you are planning to go straight on to the next tutorial Data Subset Extraction using Referential Integrity, you can keep Data Subset Extraction open. Otherwise, either Click File > Exit or click the [image] button, and then click Yes.
  14. Continue to the next section.

Executing Simple Subsetting Method

By default, the Extension Technology will attempt to insert the processed data into the target database. For testing purposes, it is possible to configure the Extension to instead write the processed data into text files as comma separated values (.csv).

Execute a simple subsetting method with Extension Technology:

  1. From a MS-DOS prompt, change to the executables directory c:\Program Files\Micro Focus\Data Express 4.0\odbc.
  2. 2. Type dxestart

    The target ODBC data store now contains reduced tables.

  3. The execution finished without errors, so you are prompted to view the log file. Type Y to view the file.
  4. Close the text file.
  5. Continue to the next section.

Verifying Results

Verify that the tables are in fact different:

  1. 1. Launch the SQL Wizard: ..\mfsql\bin\xwiz40n
  2. Notice that the tables TGT.EMPLOYEE and TGT.DEPT have been reduced from SRC.EMPLOYEE and SRC.DEPT.
Parent topic: Simple Data Subset Extraction


Data Subset Extraction using Referential Integrity

In this tutorial, you import class information using referential integrity.

Note: Sessions in this tutorial should be done in order.

This tuorial walks you through the data classification and the data subsetting phases. In this session, subsetting is based on referential integrity rules.

Begin this session only after you have completed the tutorial Simple Data Subset Extraction. You must already be logged in to Data Builder to begin.

  • Sample Session
Parent topic: Data Express for Distributed Data Stores Tutorials


Sample Session

In this session, you:

  • Create referential integrity classes
  • Create a referential integrity method
  • Export referential integrity method information to Extension Technology
  1. · Execute referential integrity method

Creating Referential Integrity Classes

In this exercise you will import referential integrity classes:

  1. In Data Builder, open the Work with Data Elements window (click [image] or access it through Environment > Work with Data Elements).
    Note: You can also access the Work with Data Elements window from the main window by selecting the file name, right-clicking to view the context menu, and then by clicking Work with Data Elements of the Selected Data Store.
  2. From the Select Data Elements tab, click the Apply Filter button to display all data stores.
    Note: To narrow down the list of data elements displayed, use the filter fields (Machine ID, Company name, and Application ID), and then click Apply Filter. To display all data elements, simply click Apply Filter.
  3. Click the Classes - Assignment tab.
  4. Click Import Class.
  5. Select the Referential Integrity import type:
  6. In the Machine ID list, specify ALLIE.
  7. In the Company list, specify DXPROJ.
  8. In the Database list, specify GSDEMO.
  9. Click Show Schemas.
  10. Click Import.
  11. Close the Import Class window.

    In the All Classes pane in the Work with Data Elements window, classes CLA0201 and CLA0202 now exist in the None Super Class.

  12. In the List of Data Store Data Elements, notice:
    • class CLA0201 is assigned to CUSTOMER.C_NO and ORDERS.C_NO
    • · class CLA0202 is assigned to ORDERS.O_NO and ITEMS.O_NO
  13. Close the Work with Elements window.
  14. If you are planning to go straight on to the next tutorial Data Subset Extraction using a Combined Data Element, you can keep Data Builder open. Otherwise, either Click File > Exit or click the [image] button, and then click Yes.
  15. Continue to the next section.

Creating a Referential Integrity Method

Create a referential integrity method:

  1. In Data Subset Extraction, open the Import Method window by clicking [image] (Import Method from Referential Integrity).
  2. Enter the following details for the method:
    Machine ID | ALLIE
    Company | DXPROJ
    Group | DX40GS
    Method (method name) | RI METHOD
    Method text (method description) | RI METHOD
    Class | CLA0201
    Database | GSDEMO
  3. Click Import.
  4. Click Properties.
  5. Click Active to activate the method.
  6. Click OK to close the Method Properties window.
  7. Click OK to close the Work with Method window.
  8. Continue to the next section.

Exporting Referential Integrity Method Information to Extension Technology

Export your referential integrity method information.

  1. Click [image] (Distributed Exporter) from the main window of Data Subset Extraction.
  2. Verify that the method RI METHOD is selected.
  3. Click Export.
  4. When prompted to continue elaboration, click Yes.
  5. Click OK.
  6. Click Close to close the Distributed Exporter window.
  7. If you are planning to go straight on to the next tutorial Data Subset Extraction using a Combined Data Element, you can keep Data Subset Extraction open. Otherwise, either Click File > Exit or click the [image] button, and then click Yes.
  8. Continue to the next section.

Executing Referential Integrity Method

By default, the Extension Technology will attempt to insert the processed data into the target database. For testing purposes, it is possible to configure the Extension to instead write the processed data into text files as comma separated values (.csv).

Execute subset extraction with Extension Technology:

  1. From a MS-DOS prompt, change to the executables directory c:\Program Files\Micro Focus\Data Express 4.0\odbc.
  2. 2. Type dxestart
    The target ODBC data store now contains the table TGT.CUSTOMER, TGT. ORDERS, and TGT.ITEMS. The contents within these tables follow the referential integrity defined in the database:
    • Only records in the TGT.ORDERS table that have the same customer number as records in the TGT.CUSTOMER table are present.
    • Likewise, only records in the TGT.ITEMS table that have the same order number as records in the TGT.ORDERS table are present.
  3. Continue to the next section.

Verifying Results

Verify that the tables are in fact different.

  1. Launch the SQL Wizard:
     ..\mfsql\bin\xwiz40n
  2. Notice that the tables TGT.ORDERS and TGT.ITEMS have been reduced.
Parent topic: Data Subset Extraction using Referential Integrity


Data Subset Extraction Using a Combined Data Element

In this tutorial, you change the selection criteria in the RI METHOD you created in the tutorial Data Subset Extraction Using Referential Integrity to use a combined data element. This is a more realistic portrayal of what you will do with RI methods - that is, supply initial criteria to the lead table to drive the referential integrity constraints throughout your method.

A combined data element is essentially a "dummy" data element that you create for ease of use when it comes to manipulating classes. Once a combined data element is created, you can perform operations against the data element like you would for other elements.

Note: Sessions in this tutorial should be done in order.

This tutorial walks you through the data classification and the data subsetting phases. In this session, subsetting is based on a combined data element.

Begin this session only after you have completed the tutorial Data Subset Extraction Using Referential Integrity. You must already be logged in to Data Builder to begin.

In this session, you:

  • Create a combined data element
  • Create a class on a combined data element
  • Associate a class with a combined element
  • Modify a referential integrity method to use a combined data element as a selection class
  • Export referential integrity method to Extension Technology
  • Execute referential integrity method
  • Creating a Combined Data Element
  • Creating a Class on a Combined Data Element
  • Associating a Class with a Combined Data Element
  • Modifying a Referential Integrity Method to Use a Combined Data Element as a Selection Class
  • Exporting a Referential Integrity Method to Extension Technology
  • Executing Referential Integrity Method
  • Verifying Results
Parent topic: Data Express for Distributed Data Stores Tutorials


Creating a Combined Data Element

Create a combined data element:

  1. 1.Launch Data Builder .
  2. Double-click the row for SRC.CUSTOMER.
  3. Use <Ctrl> to select the rows for the data element names STATE and BALANCE.
  4. Right-click to display a context menu, and then click Add to Combined Data Element.
  5. Change the Data element name to FLDSTBAL.
  6. 5. In the Description box, specify ST BALANCE COMBINED FIELD.
  7. Click Apply.
  8. Click Close to close the Add to Combined Data Element window.

    The combined data element FLDSTBAL you just created is added to the List of Data Elements for the data store SRC.CUSTOMER.

  9. Continue to the next section.
Parent topic: Data Subset Extraction Using a Combined Data Element


Creating a Class on a Combined Data Element

In this exercise, you will create the class STBAL on a combined data element.

  1. In Data Builder, open the Work with Classes window (click [image] or access it through Environment > Work with Classes).
  2. Click New.
  3. In the Name field, specify STBAL as the name of the class you want to create.
  4. In the Full description field, specify STBAL as the description of the class.
    Note: The description will be visible as a label when you assign the class to data elements within data stores.
  5. Select the Super Class None.
  6. 6. Select the data type Alpha.
  7. 7. Leave the remainder of the fields at their default values.
  8. 8. Click Apply.

    The class STBAL appears at the end of List of Classes.

  9. 9. Click OK to close the Work with Classes window.
  10. 10. Continue to the next section.
Parent topic: Data Subset Extraction Using a Combined Data Element


Associating a Class with a Combined Data Element

In this exercise you will be manually associating the STBAL class you just created with columns in table CUSTOMER:

  1. In Data Builder, open the Work with Data Elements window (click [image] or access it through Environment > Work with Data Elements).
    Note: You can also access the Work with Data Elements window from the main window by selecting the file name, right-clicking to view the context menu, and then by clicking Work with Data Elements of the Selected Data Store.
  2. Click the Apply Filter button to display all data stores.
    Note: To narrow down the list of data elements displayed, use the filter fields (Machine ID, Company name, and Application ID) to narrow down the list of fields displayed, and then click Apply Filter. To display all fields, simply click Apply Filter.
  3. Click the Classes - Assignment tab.
  4. In the All Classes pane, expand the None Super Class, and then select the class STBAL.
  5. Click the Data Store Name column to data elements by data store.
  6. In the List of Data Elements, select the row for CUSTOMER.FLDSTBAL. (That is, select the row for the data storeCUSTOMER and the name FLDSTBAL.)
  7. Click the Assign Class button to assign the STBAL class to CUSTOMER.FLDSTBAL.

    STBAL now appears in the Assigned Class column for CUSTOMER.FLDSTBAL.

  8. Click the Close button to close the Work with Data Elements window.
  9. Exit Data Builder. (Click File > Exit or click the [image] button, and then click Yes.)
  10. Continue to the next section.
Parent topic: Data Subset Extraction Using a Combined Data Element


Modifying a Referential Integrity Method to Use a Combined Data Element as a Selection Class

Modify a referential integrity method.

  1. Launch Data Subset Extraction.
  2. In the List of Methods in Group, double-click the row for the method RI METHOD.
  3. Maximize the Work with Method window.
  4. In the Data Stores Elaboration list, right-click step 10, point to Elaboration, and then click Selection class / Filter properties.
  5. Select the row for the class STBAL.
  6. Click the filter type FILTER BY RANGE:
  7. Specify MD in the From and To boxes.
  8. In the Choose a data element list, click BALANCE.
  9. Specify 0 in the From box, and 1000 in the To box.
  10. Click OK to close the Selection class / Filter properties window.
  11. Click OK to close the Work with Method window.
  12. Continue to the next section.
Parent topic: Data Subset Extraction Using a Combined Data Element


Exporting a Referential Integrity Method to Extension Technology

Export method information:

  1. Open the Distributed Exporter window by clicking from the main window.
  2. Verify that all values are populated as shown previously.
  3. Click Export.
  4. When prompted to continue elaboration, click Yes.
  5. Click OK to close the Distributed Exporter window.
  6. Exit Data Subset Extraction. (Click File > Exit or click the [image] button, and then click Yes.)
  7. Continue to the next section.
Parent topic: Data Subset Extraction Using a Combined Data Element


Executing Referential Integrity Method

Execute subset extraction with Extension Technology:

  1. From a MS-DOS prompt, change to the executables directory c:\Program Files\Micro Focus\Data Express 4.0\odbc.
  2. Type dxestart
    The target ODBC data store now contains the table TGT.CUSTOMER, TGT. ORDERS, and TGT.ITEMS. The contents within these tables follow the referential integrity defined in the database:
    • Records in the TGT.CUSTOMER table have been reduced to only contain MD records with a balance of less than $1000 (USD).
    • Records in the TGT.ORDERS table have been reduced to only contain records that have the same customer number as the parent table TGT.CUSTOMER.
    • Likewise, records in the TGT.ITEMS tables have the same order number as records in the parent TGT.ORDERS table
  3. Use standard tools and procedures relevant to your data store to verify that the information in the reduced tables is in fact different.
Parent topic: Data Subset Extraction Using a Combined Data Element


Verifying Results

Verify that the tables are in fact different:

  1. Launch the SQL Wizard:
     ..\mfsql\bin\xwiz40n
  2. Notice that the tables TGT.ORDERS, TGT.CUSTOMER, and TGT.ITEMS have been reduced.
Parent topic: Data Subset Extraction Using a Combined Data Element


Simple Sampling

In this tutorial, you perform sampling to obtain information about the distribution of data. Then you use that distribution as a way to assign classes to other data elements with similar data distributions.

Known Restriction: Data Express does not support sampling for binary data.

Note: Sessions in this tutorial should be done in order.

This tutorial walks you through the data preparation, data inventory, and data sampling phases.

Begin this session only after you configured connectivity for the sample XDB data store provided in the product installation in the tutorial Using Data Express. We recommend that you begin this session only after you have completed all previous tutorials.

You must already be logged in to Data Builder to begin.

  • Sample Session
Parent topic: Data Express for Distributed Data Stores Tutorials


Sample Session

The sample sessions are broken out into three parts: one to add the demo sampling tables to your XDB database TUTORIAL location, one to perform data inventory, and one to perform the actual sampling.

Note: Sessions in this tutorial must be done in order.

Database Preparation for Sampling Exercises

In this session, you prepare your sampling data by adding tables to your XDB database TUTORIAL location, which is necessary to complete the sampling exercises.

Note: The tables needed for this tutorial are different than the ones required for the previous tutorials.
  1. From a MS-DOS prompt, type:
     cd
          InstallationDirectory\mfsql\bin

    where InstallationDirectory specifies the directory where Data Express 4.0 is installed, for example, c:\Program Files\Micro Focus\Data Express 4.0.

  2. Load the data by executing the following command:
     xwiz40n /b sampling.sql
  3. Once the command has finished running, continue to the next section.

Data Inventory

In this session, you prepare your Data Express environment for the sampling data stores and add metadata for the sampling data stores into the Knowledge Base:

  • Set your sampling workspace
  • Load data store information into the Knowledge Base

Setting Your Sampling Workspace

Data Builder lets you set your workspace as part of the Data Inventory process. You must specify your workspace to organize and sort data on the basis of a logical model:

  1. From Data Builder, open the Work with Companies window (click [image] or access it through Environment > Work with Companies) to create a new company code.
    Note: Notice that the Machine IDALLIE is listed with the DXPROJ Company Code in the List of Companies. You created ALLIE during the tutorial Data Inventory. For the purpose of this tutorial, you create a new company under the ALLIE Machine ID. If for some reason you did not perform the Data Inventory tutorial, you must create a Machine ID prior to performing this step.
  2. Click New.
  3. Select the ALLIE Machine ID.
  4. Specify SAMPLING as the name of your company

    code in the Company Code box.

  5. In the Company Description box, type: SAMPLING EXERCISE.
  6. By default, the Life Cycle is activated. Leave this box selected.
  7. By default, the Is a Backup Company checkbox is not selected. Leave this box unselected.

    During the definition of a company, the backup company associated with it can also be defined.

    A Backup Company is useful when all the information concerning the files that undergo the Life Cycle procedure needs to be preserved.

    Note: Only one version for each file is stored in the backup company.
  8. Click Apply. The company you just created appears in the List of Companies.

  9. Click OK to save your specification and to close the window.

  10. Continue to the next section.

Loading Data Store Information into the Knowledge Base

Perform this procedure to map and load the definitions from your data stores into the XDB Knowledge Base. The information from your data stores will be mapped to your workspace.

Load data store information into the Knowledge Base:

  1. Open the Distributed Loader window by clicking [image].
  2. Select the Machine ID ALLIE and the Company SAMPLING.
  3. For this tutorial, leave the load option Skip selected.
  4. In the Source database list, select GSDEMO.
  5. In the Schema box, type: SAMPLING.
  6. In the User name box, type: install.
  7. In the Password box, type: 1.
    Note: This is the number one.
  8. Click Preview Tables to view all the available tables in the source database.
  9. Click Load to begin the export of the metadata into the Knowledge Base.
  10. Click Yes.

    An Insert tables window appears, followed by an Information window. After the metadata has fully loaded, an Information window appears that displays data store information including the number of data stores.

  11. Click OK.
  12. 12. Click Yes to display the log file in a text format.
  13. 13. Close the notepad view of the log file.
  14. 14. Click Close to close the Distributed Loader. The list of data stores in the [Work with Data Stores] window is updated automatically.
  15. 15. Keep Data Builder open, and continue to the next section.

Data Element Sampling

The data for this session is actually located in an XDB database that resides on the Windows machine where you are running your Data Express client software. Typically when this is the case, you would simply execute sampling by clicking Start in the Distributed Sampler window; there is no need to export sampling configuration, execute sampling, and import results.

In this session however, sampling is executed in a similar fashion as it would be in a UNIX environment. Therefore, sampling configuration must be exported, sampling executed, and results imported.

In this session, you:

  • Set and export sampling configuration
  • Execute sampling
  • Import sampling results
  • Verify results
  • Associate a fingerprint to a class
  • Associate a class to data elements

Setting and Exporting Sampling Configuration

Perform this procedure to verify your sampling configuration settings and to export information about the data store to output files.

Note: This exercise contains several references to the config subdirectory. For more information, see chapter Extension Technology Directories.
  1. In the List of Data Stores in the Work with Data Stores window, select a row for a SAMPLING.TableName data store.
  2. Right-click and select Enable Sampling.

    Enabling sampling changes the classification number value for the data store to a 1.

    Classification numbers are used to restrict the data stores that get sampled. By default, all data stores are assigned a Classification number of 0, which means that sampling is disabled. When sampling the data stores within your work environment, all enabled data stores with Classification number values of 1 will be sampled.

  3. Repeat steps 1-2 for each SAMPLING.TableName data store.
  4. In the List of Data Stores, select a data store with the Machine ID ALLIE and the company SAMPLING, right-click, and then click Data Store Properties:
  5. Note that the Classification number value is a 1.

    Classification numbers can be changed in the Properties window to provide further granularity in restricting which data stores get sampled as based on user-defined criteria. The data store remains enabled as long as the Classification number is not 0.However, just because sampling is enabled does not mean that the data store is a candidate for sampling as this is controlled by Classification number in the Distributed Sampler.

    If the Classification number set for the data store is less than or equal to the Classification number specified in the Distributed Sampler, the data store will be sampled. Likewise, if the data store Classification number is greater than the Classification number in the Distributed Sampler, the data store will not be sampled.

    Tip: You can assign classification numbers to your data stores based on how often you want to sample data. For example, 1 could represent daily sampling, 2 could represent weekly sampling, and 3 could represent monthly sampling.

  6. Open the Distributed Sampler window by clicking [image].
  7. Select the ALLIE Machine ID if it is not already selected.
  8. Select the SAMPLING company code.
  9. Retain all default sampling options.

    For this exercise, all data stores should be sampling candidates and all sampling options should be utilized.

    The following list describes the sampling Options:
    • Compressed sampling - Produces the data element fingerprint. The fingerprint graphically shows the distribution of values within a given range for the sampled data element. The fingerprints for numeric and alphanumeric data elements differ in that the fingerprint for an alphanumeric data element shows the distribution of values based on the first character and provides additional information.
    • Standard sampling - Displays information for each data element value including the number of times each value occurred and the percentage that value represents in the total population.
    • Min/Max calculation - Displays the minimum and maximum values for the data element.
  10. Note the Output directory specified. By default, this is the directory path to your config subdirectory, which is located in the data directory appropriate to the ODBC Extension.

    Notes:

    • By default, the directory path to the config subdirectory is listed for the ODBC Extension. If you are using the Oracle Extension, this must be changed.
    • You can change the Output directory to any directory; just make sure that the files sampling.dat and method.rc are copied to the appropriate config directory prior to executing sampling.
  11. Click Advanced.
  12. Retain all default values.
    Note: The Data Element Size of 0 (zero) means to include all sizes in the sampling results and does not actually reflect the value of 0 when Ignore Special Values Zero / Space is checked. Likewise, the default 0 values in Max. And Min. Recalculation Additional Options mean to consider all values to be in-range and does not reflect the value of 0.
  13. Click OK to return to the Distributed Sampler window.
  14. Click Export:
  15. Click OK.
    Two output files have been created and are located in the output directory on your Windows machine:
    • sampling.dat - contains information regarding the data elements to be sampled
    • method.rc - contains encrypted connectivity information about the source data store (data store name, username, and password)
  16. Verify that the files sampling.dat and method.rc are located in the config subdirectory.

    Important: Under normal circumstances, you would need to copy the files sampling.dat and method.rc to the config subdirectory on the machine where your source data store is located. However, in this exercise the files are already in the correct location.

  17. Continue to the next section.

Executing Sampling

During sampling execution, the Extension Technology uses the output files sampling.dat and method.rc to produce the sampling.log file that contains the resulting statistics.

  1. From a MS-DOS prompt, change to the Extension Technology executables directory. For example, change to c:\Program Files\Micro Focus\Data Express 4.0\ODBC.
    Note: For more information about directories, see Extension Technology Directories.
  2. Type dxesampling.
  3. Change to the log directory.
    Note: For more information, see section Executables in Extension Technology Directories.
  4. Verify that the sampling.log file is listed.
  5. Continue to the next section.

Loading Sampling Results

The sampling.log file that was generated during sampling execution must be loaded into the Knowledge Base.

  1. Open the Distributed Sampler window by clicking [image].
  2. Verify that the Machine ID selected is ALLIE.
  3. In the Company list, click SAMPLING.
  4. Click the Load results from file tab.
  5. Click [image] and browse for the sampling.log file that was generated during sampling execution.
  6. Click Load.
  7. Continue to the next section

Verifying Results

Once you have exported sampling configuration information, executed sampling, and loaded the results into the Knowledge Base, you can view the results.

When compressed sampling is performed, a fingerprint is created. The fingerprint is a unique graphical representation of the distribution of data for the sampled data element. When the results for compressed sampling are loaded, the fingerprint is created.

  1. In Data Builder, click the Sampling tab:
  2. Select the row for the SAMPLING.TABADD data store name in the List of data stores - Sampling Active list.
  3. Click [image] (Show Synthetic Data Elements) or right-click and then select Show Synthetic Data Element Contents.

    The Data Store Data Elements list in the left pane shows the data elements for the data store you selected.

  4. In the Data Store Data Elements list, select the data element PRGADDR.

    You can also select the image for the data element in the Data Element Samples grid. To ensure that you have selected the correct one, the related data element name is highlighted in the Data Store Data Elements list.

    Sampling results for the numeric data element PRGADDR are displayed.

  5. Review the data element fingerprint in the Zoomed Data Element Sample. The fingerprint is the result of performing compressed sampling.
    Notice the following:
    • The numbers on the vertical y-axes represent the ranges of values based on the decimal place, where each number represents one decimal place to the left of the decimal point. For instance, 0 represents the ones place (0-9), 1 represents the tens place (10-99), 2 represents the hundreds place (100-999), and so on.
    • The numbers on the horizontal x-axes represent a specific range based on the corresponding y-coordinate. For instance, 1 represents the value 1 for the y-coordinate 0, while 1 represents values 10-19 for the y-coordinate 1.
    • The values in each cell indicate the number of times a value in the data element falls within the specified range.
    Note: The fingerprint can also be viewed as a bar graph by clicking Graph.
  6. Review the items in the list Sample Analysis of the Selected Data Element. This list shows the actual data element values when standard sampling was performed.

    Known Restriction: Currently Data Express only shows the first 1000 distinct values and ranges, which are sorted either numerically or alphanumerically based on the data element type. When there are more than 1000 distinct data element values, the Data Element Value ends at the 1000th unique value instead of the greatest data element value.

  7. In the Data Store Data Elements list, select the data element ADDRESS.

    Sampling results for alphanumeric data element ADDRESS are displayed.

    Review the data element fingerprint in the Zoomed Data Element Sample. The fingerprint is the result of performing compressed sampling.

    Notice that the fingerprint for an alphanumeric data element contains four distinct sections:
    Section Type Example Description

    Character

    Distribution

    A range of characters in alphabetical order is provided to show how many data elements begin with that character.

    Number

    Distribution

    A range of numbers is provided to show how many data elements begin with the indicated number. In the provided example, no data elements begin with numbers.

    Type Summary

    An alphanumeric data element can actually be a numeric-only data element or alphanumeric data element. The type summary provides a count of how many data elements fall into either category. In the provided example, all data elements are alphanumeric in type.

    Field Length

            The numbers on the vertical y-axes represent the ranges of values specified. The numbers on the horizontal x-axes represent the position of the actual number in that range. For instance, 90 in the provided example means that there are 90 values with a length equal to 11.
    Note: The fingerprint can also be viewed as a bar graph by clicking Graph.
  8. Close the Show Synthetic Data Element Contents window to return to Work with Data Stores.
  9. Open the Work with Data Elements window (click [image] or access it through Environment > Work with Data Elements).
  10. In the Machine ID list, click ALLIE.
  11. In the Company name list, click SAMPLING.
  12. Click Apply Filter. Work with Data Elements - Sampling Tables
  13. Note the Minimum Value and Maximum Value items in the List of Data Elements, which are the result of performing min/max calculation sampling.
  14. Leave the Work with Data Elements window open, and continue to the next section.

Creating a Sampling Class

You can create a new class to be used for sampling purposes, or you can use a predefined class. In this exercise you will create a sampling class named SAMPNUM.

  1. Open the Work with Classes window (click [image] or access it through Environment > Work with Classes).
  2. Click New.
  3. In the Name box, specify SAMPNUM as the name of the class you want to create.
  4. In the Full description field, specify SAMPLING NUMERIC VALUES as the description of the class.
    Note: The description will be visible as a label when you assign the class to data elements within data stores.
  5. Select the Super Class Numbers.
  6. Select the data type Numeric.
  7. Leave the remainder of the fields at their default values. Click Apply.

    The class SAMPNUM is now listed in the List of Classes.

  8. Click OK to return to the Work with Data Elements window.
  9. Click Refresh.
  10. Continue to the next section.

Associating a Fingerprint to a Class

In Data Express, you can correlate data elements if their value distributions are similar. To do this, you must first associate a fingerprint, which represents the desired distribution, to a class.

This fingerprint becomes the prototype that is used determine the class assignment for other data elements. If the prototype and the fingerprint for a data element are deemed similar based on a calculated confidence value, the data element is also associated with the prototype class.

  1. Click the Classes - Assignments tab.
  2. In the All Classes pane, expand the Number Super Class, and select the class description SAMPLING NUMERIC VALUES.
  3. In the List of Data Elements, select the row for the data element PRGNAME in the data store TABFNAM.
  4. Check the Selected Data Element Attributes check box:
  5. Drag the fingerprint that now appears beneath the text Selected Data Element Attributes, to the first Class Samples cell.

    Work with Data Elements - Associated Class Fingerprint (Prototype)

    By highlighting the SAMPNUM class description in the All Classes pane and then by dragging the fingerprint for the PRGNAME data element to the Class Samples cell, the SAMPNUM class is then assigned to the fingerprint. This fingerprint becomes the prototype. This action also associates the SAMPNUM class to the PRGNAME data element.

  6. Continue to the next section.

Associating a Class to Data Elements

In this exercise you are associating the class SAMPNUM (which is associated to your sampling prototype) to other data elements with sampling data distributions similar to the prototype fingerprint.

The level of similarity between distributions depends on the thresholds you set when importing the class information. If the prototype and the fingerprint for a data element are deemed similar based on a calculated confidence value, the data element is also associated with the prototype class.

  1. From the Work with Data Elements window, click Import Class. The Import Class window is displayed.
  2. Click Sampling Results.

    When comparing the fingerprint for a data element to the prototype fingerprint, internal Data Express confidence values are calculated to provide measurements that illustrates the similarities between the data distributions. The internal confidence values are used as input for the threshold formulas.

  3. In the Machine ID list, click ALLIE.
  4. In the Company list, click SAMPLING.
  5. In the Super Classes section, click Numbers.
  6. Set the Theshold 1 value to 82.

    The value for Threshold 1 represents a percentage where the similar areas in the two fingerprints are weighted more heavily than the dissimilar areas.

  7. Keep the logical operator set to AND.
  8. Set the Theshold 2 value to75.

    The value for Threshold 2 represents a percentage where all areas (both similar and dissimilar) in the two fingerprints are given equal weight.

  9. Click Import:
  10. Click OK.
  11. Close the Import Class window and return to the Work with Data Elements window:
  12. Click the Close button to close the Work with Data Elements window.

    A sampling class has successfully been assigned to data elements with similar distributions to the prototype.

  13. Close Data Builder; either Click File > Exit or click the [image] button, and then click Yes.
Parent topic: Simple Sampling


Data Generation

In this tutorial, you perform basic data generation, working with the TGT.DEPT and TGT.EMPLOYEE tables to populate a table based on the content of another table and on data in a dictionary.

Begin this session only after you have set your workspace and loaded data store information into the Knowledge Base by following the instructions in tutorials Using Data Express and Data Inventory. You must be logged into Data Builder before you begin.

In this session, you:

  • Catalog the XDB tables provided in the sample
  • Start the Data Generator module.
  • Create simple Data Generation code
  • Submit JCL to generate data

Complete these tutorials in the following order:

  1. Catalog and Classify Tables
  2. Start the Data Generation Module
  3. Create Simple Data Generation Code
  4. Execute a Simple Data Generation Law
  5. Verify Results
Parent topic: Data Express for Distributed Data Stores Tutorials
Parent topic: Data Express for z/OS Tutorials


Catalog and Classify Tables

  1. Using a different machine and company, repeat the cataloguing phase, but loading only the TGT.DEPT and TGT.EMPLOYEE tables.
  2. Execute the classification phase as described in the Data Subset Extraction part of this guide, assigning the same class to the DEPTNO columns of both TGT.DEPT and TGT.EMPLOYEE.
Parent topic: Data Generation
Next topic: Start the Data Generation Module


Start the Data Generation Module

  1. Start the Data Generation module from your Start menu or tile, depending on your Windows operating system.

    Note: If you are using Data Express on Windows Vista and User Access Control is enabled, you must run the Data Generation module as Administrator.

  2. Select the XDB Knowledge Base as the required database for your ODBC connection.

    By default, the highlighted item in the list of available databases is the last database you connected to using Data Masking. This becomes apparent after you click Available databases.

  3. Click OK.
  4. Log in using your username and password.
  5. Click OK.
  6. Select the schema DX40; click OK.
Parent topic: Data Generation
Previous topic: Catalog and Classify Tables
Next topic: Create Simple Data Generation Code


Create Simple Data Generation Code

  1. Launch the New Code Wizard from the New Code icon.
  2. Specify the proper machine and company.
  3. Give the code the name SIMPLEC and provide a description.
  4. Click Apply.

    This creates the code.

  5. Select the code, and look at the lower part of the screen; it shows the laws that are used in the code, one law per table.
  6. Ensure that the TGT.EMPLOYEE table appears before the TGT.DEPT table. If now, reorder the tables using the arrows on the right.
  7. Right-click on TGT.EMPLOYEE; then select Set directive from the context menu.
  8. Select the DEPT column; then right-click and select Assign output class specifying the DEPT column.
  9. Right-click TGT.EMPLOYEE; then select Don’t create output on the context menu.
  10. Right-click TGT.DEPT; then select Set directive from the context menu.
  11. Right-click the COD_CUS column; then select Set directive from the context menu, specifying Other file dependencies as the directive type.
  12. Right-click TGT.EMPLOYEE; then select Set directive on the context menu.
  13. Select the DNAME column; then select Set directive from the context menu, specifying Dictionary as the directive type and USDCHSUR as dictionary name.
  14. Export the information about the tables DEPT and EMPLOYEE using the Distributed Exporter.
Parent topic: Data Generation
Previous topic: Start the Data Generation Module
Next topic: Execute a Simple Data Generation Law


Execute a Simple Data Generation Law

By default, Extension Technology attempts to insert the processed data into the target database.

Execute a simple data generation law with Extension Technology:

  1. From a command prompt, change to the executables directory, which by default is %PROGRAMFILES%\Micro Focus\Data Express 4.0\Synthetic\ODBC.
  2. Enter dxestart.

    This populates the tables in the target ODBC data store using the specified data generation laws. After the execution finishes without errors, you are prompted to view the log file.

  3. Type Y to view the file.
  4. Close the text file.
  5. Continue to the next section.
Parent topic: Data Generation
Previous topic: Create Simple Data Generation Code
Next topic: Verify Results


Verify Results

Verify that the tables are populated according to definition:

  1. Launch the SQL Wizard: ..\mfsql\bin\xwiz40n
  2. Verify that the TGT.EMPLOYEE table contains its original values, and the TGT.DEPT table has been populated according to the specified laws.
Parent topic: Data Generation
Previous topic: Execute a Simple Data Generation Law


Oracle Alias Creation

In order to use the Oracle Extension, you must first create an alias in the Borland Database Engine (BDE) for your source data store to communicate with the Oracle client installed on your machine. A BDE alias is not required for your target data store.

Note:

  • For your source Oracle data store specified for either the Distributed Loader or Distributed Exporter, you must create a BDE alias. We recommend that the associated BDE alias match the Oracle TNS service name specified in your tnsnames.ora file.
  • For your target Oracle data store specified in the Distributed Exporter, make sure that the target name is the Oracle TNS service name specified in your tnsnames.ora file located on the machine where you intend to mask/subset data.
  • If you are using Data Express on Windows Vista and User Access Control is enabled, you must run the BDE Administrator utility as Administrator.

To create an Oracle alias:

  1. Open BDE Administrator (from the Control Panel). Click the Configuration tab, and then select ORACLE from the Native Drivers list as shown in Figure A-1:
  2. In the Definition of ORACLE pane, click in the second column for DLL32 to display a list of DLLs. Select SQLORA8.DLL, and click Object > Apply as shown in Figure A-2:
  3. 3. In the Definition of ORACLE pane, click in the second column for VENDOR INIT, and enter OCI.DLL as the Vendor DLL file name.
  4. 4. In the Drivers and System pane, click the Databases tab.
  5. 5. Create a new Oracle database alias (right-click, select New, select the Database Driver Name ORACLE, and then OK) and rename the alias as appropriate.
  6. 6. In the Definition of AliasName pane, click in the second column for SERVER NAME, and enter your Oracle TNS.
Parent topic: Getting Started with Distributed Data Stores


Oracle Wallet

Oracel Wallet allows the Data Express engine to connect the source and target databases without needing to specify the user name and password.

Oracle Wallet is a container/repository that stores credentials such as certificates, certificate requests, and private keys. You can use this feature if you enable HTTPS. Oracle Wallets can be stored either on the file system or an LDAP Server (like OID).

You manage Oracle Wallets using Oracle Wallet Manager (OWM - java based application). OWM is executable in $ORACLE_HOME/bin .

Using OWM you can:

  • Create wallets
  • Create certificate requests
  • Import certificates to wallets
  • Upload wallets to a directory
  • Implementing Oracle Wallet Support
    Describes how to implement Oracle Wallet support.
Parent topic: Getting Started with Distributed Data Stores


Implementing Oracle Wallet Support

There is a new parameter in the dxeconfig.cfg file named Identify Externally that identifies the connection criteria. This parameter is at the end of the file in the section applicable to the Oracle Extension only.

The possible values are Y and N. Default value is N.

Yes
Data Express will skip the reading of user name and password from method.rc. In this case, the connection to the database, instead of happening in the usual way (using the user and password specified during the “export” phase), will use the configured Wallet (if the Wallet has been configured properly). The system will connect to the database without needing explicit user name and password credentials.
No
Data Express behavior will remain unchanged and access continues to require a valid user name and password.

The user interface will remain unchanged (and in case of use of wallet feature, the data specified as user and password are ignored ), and the only change needed, from the user perspective, is that in order to use the new feature, you must set the dxeconfig.cfg configuration file properly.

Here is a sample of the dxeconfig.cfg file:

* Data Express Configuration File

LogLevel:1
Always Drop Table:N
CSV Data Store:N
COBOL Masking:Y
Validate Target Table:Y
Validate Source Schema:N
Enable Logging:N
Hide Progress:N

* These entries are only applicable to the Oracle Extension.

Text Output:N
Data Separator:,
Fields Enclosed By:"
Identify Externally:Y

The dxeconfig file is not used for sampling. In order to use wallet management, you need to specify a –e parameter.

Use the following command to use identification via user and password: Dxestart

Use the following command to use identification via wallet: Dxestart -e

Parent topic: Oracle Wallet


Getting Started with z/OS Data Stores

Contains explanatory information and a series of tutorials that show you how to start using the product in order to analyze and manage data stores from System z servers running z/OS.

The tutorials introduce the major features in a number of short sessions, typically 10 - 30 minutes.

This guide is intended for customers who are using z/OS data stores. If you have the Data Express for z/OS solution but have also purchased the additional ODBC Extension or Oracle Extension, use the Getting Started with Distributed Data Stores Guide instead.

  • Product Components
    Describes the main components of the product.
  • Using z/OS Data Stores
    Describes the high level flow of using z/OS data stores.
  • Data Express for z/OS Tutorials
  • Managing XML and Large-object Columns
    Describes how to manage masking and subsetting of XML data stored in XML or LOB/BLOB/CLOB columns of z/OS DB2 tables, supporting both DSNTIAUL and UNLOAD formats.
Parent topic: Data Express
Related concepts
Catalog z/OS Data


Product Components

The following is a brief description of the major components of Data Express:

Data Builder
The core Data Express module that enables you to take inventory of your data, and store information in the Data Express Knowledge Base, which resides on z/OS. Data Builder works with both Data Masking and Data Subset Extraction.
Data Masking
The module that enables you to generate a test environment where personal and sensitive data has been made anonymous by using predefined or customized masking routines.
Data Subset Extraction
The module that enables you to generate a test environment where a consistent and congruent subset of data has been extracted to allow for a manageable volume of data.
Workspace
A two-dimensional working environment definition from which you can categorize and classify data stores for your test environment.
Data Express Knowledge Base
A collection of tables within a database that contain all information about catalogued files, including properties and processing rules. The Knowledge Base for the Data Express for z/OS solution is a Mainframe DB/2 repository located in your z/OS environment.
Parent topic: Getting Started with z/OS Data Stores


Using z/OS Data Stores

This section provides a high-level overview of the work-flow for the Data Express for z/OS solution. For more information, see the . The work-flow process for z/OS data stores looks like this:

  • Data Inventory Phase (Data Builder, Work with Jobs)
    • Non-relational data stores
      • Load and analysis of the copybook
      • Load, analysis, and association of the data store
    • Relational data stores
      • Load DB2 or ADABAS tables
  • Data Classification Phase (Data Builder, Work with Jobs)
    • Manual Association
    • Import Classification from Referential Integrity
    • Import Classification from Data Dictionary
    • Data Analysis
      • Sampling
      • Fingerprinting
  • Data Masking Phase (Data Masking)
    • Method creation (not mandatory and only used in conjunction with Data Subset Extraction to create a test environment)
    • BTUNLDR - Create Data Changer Jobs (Data Builder, Work with Jobs)
  • Data Subsetting Phase (Data Subset Extraction)
    • Method creation
      • BTEDDUR - Import Method From Referential Integrity job (Data Builder, Work with Jobs)
      • Create New Method Wizard (Data Subset Extraction)
      • Data Masking module
    • Test Environment Creation (Data Builder, Work with Jobs )
  • Life Cycle (Data Builder)
Parent topic: Getting Started with z/OS Data Stores


Data Express for z/OS Tutorials

These tutorials show you how to prepare your Data Express working environment in z/OS and introduce the Data Express Windows client, which consists of three Data Express modules: Data Builder, Data Masking, and Data Subset Extraction. Follow the tutorials in this guide in order after you have completed your product installation.

The easiest way to learn about Data Express is to complete a few simple tasks. This guide takes you through configuring Data Builder, loading metadata into the Data Express Knowledge Base, using Data Masking to implement masking rules on a small amount of data, using Data Subset Extraction to create a simple subset of data, and finally using Data Subset Extraction twice more to create subsets of data using referential integrity. Please note that some of the data reduction examples also mask data, showing that Data Express can do both functions in one method invocation.

The three modules, Data Builder, Data Masking, and Data Subset Extraction, and tutorial files contain all the tools you need to perform these tasks.

This tutorial describes the process of configuring your z/OS environment so that you can use the Data Express Windows client (as demonstrated in subsequent tutorials) for data masking and data reduction.

Machine IDs and Company Codes

A Machine ID and Company code pair is essentially an identifier for your Data Express working environment or workspace. The Machine ID is the higher-level identifier, while the Company code must be identified by a Machine ID. You can create a new Machine ID and a new Company code for each test environment, or you can create new Company codes under a single Machine ID as needed for each test environment you want to create.

Note: A single machine ID can correspond to numerous company codes. A new company code can be created only if an existing machine ID is first selected.

Sample Data

Data Express includes the files that create the sample DB2 tables and sequential files that are used throughout the exercises in this tutorial . The database and its sample tables must be created and populated after product installation.

Note: The JCL job streams, files, and scripts needed to create your tutorial environment are located in the tutorial folder of your installation directory.

In this session, you load data needed for this tutorial into your mainframe environment and you use Data Builder to define your workspace. This involves editing several files, executing programs, creating a machine ID and company code pair for your workspace, and submitting jobs that will load information into your mainframe environment from external files.

You can follow the tutorials for either DB2 data stores, sequential files or both DB2 data stores and sequential files.

  • Sample Sessions
    Describes the sample sessions tutorial.
  • Data Masking
    Describes data masking.
  • Simple Data Subset Extraction
    Describes simple data subset extraction.
  • Data Subset Extraction using Referential Integrity and Combined Data Elements
    Describes how to do data subset extraction using referential integrity and combined data elements.
  • Data Element Sampling
    Describes data element sampling
  • Data Generation
    The Data Generation module is used to generate a test environment.
Parent topic: Getting Started with z/OS Data Stores


Sample Sessions

The sample sessions are broken out into two parts:

  • DB2 data stores
  • sequential files
Note: Sessions in this tutorial must be done in order. If you choose to complete the tutorial for both DB2 and sequential data, then do all sessions. If you choose to complete the tutorial for only DB2 or sequential data, then follow the sessions only for that data type.
  • DB2 Data Stores Session
    Describes the DB2 data stores session.
  • Sequential Files Session
    Describes the sequential files session.
Parent topic: Data Express for z/OS Tutorials


DB2 Data Stores Session

In this session, you prepare your z/OS environment for DB2 data stores:

  1. Import JCL job streams into your mainframe environment.
  2. Create DB2 tablespace and tables.
  3. Start Data Builder.
  4. Set your workspace.
  5. Load data store information from an external interface.
  • Importing JCL Job Streams into Your Mainframe Environment
    Describes how to import JCL job streams into your mainframe environment.
  • Creating the DB2 Tablespace and Tables
    Describes how to create the DB2 tablespace and tables.
  • Starting Data Builder
    Describes how to start data builder.
  • Setting Your Workspace
    Describes how to set your workspace.
  • Loading Data Store Information into Your Mainframe Environment
    Describes how to load data store information into your mainframe environment.
Parent topic: Sample Sessions


Importing JCL Job Streams into Your Mainframe Environment

In order to import the JCL CREATBT and CREASEQD into your mainframe environment, you must first edit an FTP script, PUT_JCL_DB2, to include your login credentials.

Next, you run FTPJCLDB2.BAT, which executes the FTP commands listed in the file PUT_JCL_DB2.txt to transfer the JCL job streams to an existing library in your mainframe environment.

To import the JCL job streams CREATBT and CREASEQD into your mainframe environment:

  1. Edit the FTP script in the PUT_JCL_DB2.TXT file with the following information:
    123.123.123.123   the IP address of your mainframe
YOURUSERNAME      your mainframe user name
YOURPASSWORD      your mainframe password
YOURJCLLIBRARY    the name of the library that you want to contain the JCL

    This FTP script transfers the JCL job streams CREATBT and CREASEQD onto the mainframe environment into your the specified JCL library when FTPJCLDB2.BAT is executed.

  2. Ask the DBA to customize the following parameters in the CREATBT.TXT file:
    • DSN
    • DSN SYSTEM
    • PLAN
    • LIB
    • TABLESPACE
    • DATABASE NAME
    • STOGROUP
    • PRIQTY
    • SECQTY
    • FREEPAGE
    • PCTFREE
    • BUFFERPOOL
    • SEGSIZE
    • COMPRESS

    The CREATBT.TXT file contains the JCL that is used to create the tablespace DETESTLG and the following DB2 tables:

    • CUSTOMER
    • ACCOUNT
    • CCARD
    • OPERAT
  3. Change each instance of the XXXXXX.XXXXXX library in the CREASEQD.TXT file to URADAR.DEMOT. For example, DSN=XXXXXX.XXXXXX.LISTDB2 becomes: DSN=URADAR.DEMOT.LISTDB2.
    Note: The library URADAR.DEMOT is used as an example throughout this tutorial. You can use any library that suits your mainframe library naming conventions. However, it is assumed that the mainframe user ID you are using has no restrictions and can therefore access this library.
  4. Execute FTPJCLDB2.BAT to import the JCL job streams into your mainframe environment.
  5. Continue to the next section.
Parent topic: DB2 Data Stores Session


Creating the DB2 Tablespace and Tables

In this exercise, you submit the CREATBT JCL (to create the tablespace and tables) and the CREASEQD JCL needed to complete your environment creation. To create the DB2 tablespace and tables:
  1. Within your TSO environment, submit the JCL job streams CREATBT and CREASEQD.
  2. Review results to verify that the jobs ran successfully.
  3. Edit the FTP script in the PUT_FILE_DB2.txt file with the following information:
    123.123.123.123    the IP address of your mainframe
YOURUSERNAME		     your mainframe user name
YOURPASSWORD			    your mainframe password
XXXXXX.XXXXXX      the name of the library created by JCL CREASEQD, for example: URADAR.DEMOT.

    This FTP script creates the files LISTDB2 and REFINTEG specified by the CREASEQD.TXT JCL.

    Note: The library URADAR.DEMOT is used as an example throughout this tutorial. You can use any library that suits your mainframe library naming conventions. However, it is assumed that the mainframe user ID you are using has no restrictions and can therefore access this library.
  4. Within your Windows environment, execute FTPTUTORIALDB2.BAT.BAT to import the informaiton into your mainframe environment.
  5. Continue to the next section.
Parent topic: DB2 Data Stores Session


Starting Data Builder

Data Builder enables functionality for the Data Masking and Data Subset Extraction modules. With Data Builder, you can take an inventory of data, and classify the data as you see fit. These processes store information about the data in the Data Express Knowledge Base repository.

Note: You will use the settings shown in this section throughout each subsequent chapter of this Getting Started guide.
  1. Click Start > All Programs > Micro Focus Data Express 4.0 > Data Builder.
  2. Select the mainframe Knowledge Base as the required database with which to launch the connection. In this example, the database is MVSDB2. By default, the highlighted item in the list of available databases is the last database you connected to using the Data Builder module. However, for the very first connection, MVSDB2 – OWUR40SV is selected by default. Note: By selecting MVSDB2 – OWUR40SV, you do not need to choose the database and then the schema OWUR40SV.
  3. Click OK.
  4. Log in using your mainframe user name and password.

    The [Work with Data Stores] window provides a list of files cataloged using different sets of search criteria, including:

    • Machine identifiers
    • Company names
    • Application identifiers
    • Data store types
    • Data stores with assigned classes (Multiple class Selection)
    • Data stores with assigned copybooks
    • Data store with sampling activated
    • Data stores with Data Changer activated
    • Data stores with Data Changer inactivated
    • Data stores set as Register Table
    • Data stores without Register Table attribute

    This window is comprised of four tabs: Structure, Classes, Data Changer, and Register Tables. You can select a tab to view files using a specific set of criteria as listed previously. Each tab is associated with different grouping criteria.

    The list of displayed data stores can be further restricted by applying filters. Filters can be selected from the context menu displayed by right-clicking the List of Files grid, or by selecting the specific icons displayed on the left side of the screen.

    Note: The Data Masking and the Data Subset Extraction modules are launched as separate applications.
  5. Continue to the next section.
Parent topic: DB2 Data Stores Session
Parent topic: Sequential Files Session


Setting Your Workspace

Data Builder lets you set your workspace as part of the Data Inventory process. You must specify your workspace to organize and sort data on the basis of a logical model.

Note: If you have already set your workspace for your sequential files environment, you do not need to perform this exercise. Both DB2 and sequential file data stores share the same workspace for this sample tutorial.
  1. From Data Builder, open the Work with Machine IDs window (click or access it through Environment > Work with Machine IDs) to create a new Machine ID.
  2. Click New.
  3. Specify your name as the name of the machine ID in the Machine ID box. For this tutorial, type: DATAEXPRES.
  4. Type a description of your machine ID in the Description box. For this tutorial, type: DEMO ENVIRONMENT.
  5. Specify additional information if appropriate. For this tutorial, the additional fields have been left blank. Click Apply.
  6. Click OK to save your specification and to close the window.
  7. From Data Builder, open the Work with Companies window (click or access it through Environment > Work with Companies) to create a new company code.
  8. Click New.
  9. Select the DATAEXPRES Machine ID you just created.
  10. Specify DEMO as the name of your company code in the Company Code box.
  11. In the Company Description box, type: DX GETTING STARTED GUIDE.
  12. By default, the Life Cycle is activated. Leave this box selected.
  13. By default, the Is a Backup Company checkbox is not selected. Leave this box unselected.

    During the definition of a company, the backup company associated with it can also be defined. A Backup Company is useful when all the information concerning the files that undergo the Life Cycle procedure needs to be preserved.

    Note: Only one version for each file is stored in the backup company.
  14. Click Apply. The company you just created appears in the List of Companies.
  15. Click OK.
  16. Continue to the next section.
Parent topic: DB2 Data Stores Session
Parent topic: Sequential Files Session


Loading Data Store Information into Your Mainframe Environment

Perform this procedure to map and load the definitions from your data store into the Knowledge Base. The information from your data store will be mapped to your workspace.

  1. In your mainframe environment, execute the following from an ISPF command line: TSO EXEC ‘xxxxxxxx.xxxxxxxx.CLIST(MFDATA)’ where xxxxxxxx.xxxxxxxx is a part of the PDS library name where member MFDATA is located. In both cases, this command allocates PDS libraries to ISPF and launches the Main Menu in an ISPF environment.
  2. Enter the following command: TSO CUINFIL.
  3. Select the Machine ID and Company you created.
  4. Specify the following parameters:
    Data Set Name Inp
    Name of the input data set that contains the table and owner names. For this demo, use URADAR.DEMOT.LISTDB2.
    Data Set Name Out
    the name of the output data set that contains the table names, column names, and column descriptions. For this demo, use URADAR.DEMOT.LISTODB2.
    Note: The names for input and output datasets must be different.
    Process Id
    Process identifier associated with the elaborated file. For example, for a DB2 data store with direct access, use DB2DA.
  5. Open the [Work with Jobs] window by clicking Environment > Work with Jobs to schedule a new job.
  6. Click New.
  7. In List of Jobs, select the job BURLFIL (Load Data Store Information From External Interface).
  8. Select the DATAEXPRES Machine ID you just created.
  9. Specify DEMO as the name of your company code in the Company name box.
  10. Click Apply.
  11. In the Sequential data store box, enter: URADAR.DEMOT.LISTODB2.
  12. Click OK. The job appears in the List of Scheduled Jobs. The pause symbol indicates that the job is awaiting execution. Once the job has completed successfully, the job is listed with a green check mark.
  13. In your TSO environment, submit the BURLFIL job by using the Submit Client Scheduled Job panel. The Submit Client Scheduled Job panel is accessible by selecting option Submit Client Scheduled Job from the Data Express for z/OS - Main Menu panel. For more information, see the Submission Function in the Process Guide for z/OS.
  14. In the Work with Jobs area, click the Refresh button to verify that the jobs completed successfully.
  15. Close the Work with Jobs window.
  16. In the [Work with Data Stores] window, expand the Machine ID node, and select DATAEXPRES, to view data stores associated with the DATAEXPRES machine ID:
  17. If you're planning to go straight on to the next session, you can keep Data Builder open. Otherwise, either click Exit and then Yes.
  18. Continue to the next section if you want to complete the sequential files session. Otherwise, go to the Data Masking.
Parent topic: DB2 Data Stores Session


Sequential Files Session

In this session, you prepare your z/OS environment for sequential files:

  • Import JCL CREASEQS into your mainframe environment.
  • Import the copybook and sequential file information into your mainframe environment.
  • Start Data Builder.
  • Set your workspace.
  • Load copybook information from an external interface.
  • Load data store information from an external interface.
  • Importing the JCL CREASEQS into Your Mainframe Environment
    Describes how to import the JCL CREASEQS into your mainframe environment.
  • Importing the Sequential File and Copybook Information into Your Mainframe Environment
    Describes the process for importing the sequential file and copybook information into your mainframe environment.
  • Starting Data Builder
    Describes how to start data builder.
  • Setting Your Workspace
    Describes how to set your workspace.
  • Loading Copybook Information into Your Mainframe Environment
    Describes how to load copybook information into your mainframe environment.
  • Loading Data Store Information into Your Mainframe Environment
    Describes how to load data store information into your mainframe environment.
Parent topic: Sample Sessions


Importing the JCL CREASEQS into Your Mainframe Environment

In order to import the JCL CREASEQS into your mainframe environment, you must first edit an FTP script, a JCL, list of copybook files, and a list of sequential files. The FTP script, PUT_JCL_SEQ, must be modified to include your login credentials so that you can transfer the JCL CREASEQS to an existing library in your mainframe environment. Then, the CREASEQS JCL must be modified to list the name of the library that will contain it.

After editing the provided files, you run FTPJCLSEQ.BAT, which executes the FTP commands listed in the file PUT_JCL_SEQ.txt.

  1. Edit the FTP script in the PUT_JCL_SEQ.TXT file with the following information:
    123.123.123.123  the IP address of your mainframe
YOURUSERNAME		   your mainframe user name
YOURPASSWORD			  your mainframe password
YOURJCLLIBRARY   the name of the library that you want to contain the JCL

    This FTP script transfers the JCL CREASEQS onto the mainframe environment into your the specified JCL library when FTPJCLSEQ.BAT is executed.

  2. Change each instance of the XXXXXX.XXXXXX library name in the CREASEQS.TXT file to URADAR.DEMOT.
    Note: The library URADAR.DEMOT is used as an example during instructional steps throughout this tutorial. You can use any library that suits your mainframe library naming conventions. However, it is assumed that the mainframe user ID you are using has no restrictions and can therefore access this library.

    The CREASEQS.TXT file contains the JCL that is used to create the PDS library COPYBOOK and the libraries for the following sequential files:

    • CUSTOMER
    • PHCONTRA
    • PHNUMBER
    • PHONBILL
    • EXTSEQ
    • EXTCPY
    Note: The library name URADAR.DEMOT used in this JCL must also be inserted into the FTP job within the PUT_FILE_SEQ.TXT file.
  3. Execute FTPJCLSEQ.BAT to import the JCL into your mainframe environment.
  4. Continue to the next section.
Parent topic: Sequential Files Session


Importing the Sequential File and Copybook Information into Your Mainframe Environment

In this exercise, you submit the CREASEQS JCL to create the library URADAR.DEMOT in order to complete your environment creation. Then, you import sequential file and copybook into your mainframe environment.
  1. Edit the FTP script in the PUT_FILE_SEQ.txt file with the following information:
    123.123.123.123   the IP address of your mainframe
YOURUSERNAME		    your mainframe user name
YOURPASSWORD			   your mainframe password
XXXXXX.XXXXXX     the name of the library created by JCL CREASEQS, for example: URADAR.DEMOT.
  2. This FTP script inserts the COPY and RECORD lines of the sequential files into the libraries created by the CREASEQS.TXT JCL.
  3. Within your TSO environment, submit the CREASEQS JCL.
  4. Review results to verify that the job ran successfully.
  5. Within your Windows environment, execute FTPTUTORIALSEQ.BAT to import the information into your mainframe environment.

    The information from the following sequential files is loaded into the output dataset you specified:

    Sequential File Related Data Element File Related Copybook File Data Elements
    CUSTOMER CUSTOMER_REC CUSTOMER_COPY
    • COD-CUS (customer code)
    • NAME (customer name)
    • SURNAME (customer surname)
    • DATE-OF-BIRTH (customer birth date)
    • PLACE-OF-BIRTH (customer place of birth)
    PHCONTRA PHCONTRA_REC PHCONTRA_COPY
    • COD-CUS (customer code)
    • OPERAT-NUM (operator number)
    • CONTR-NUM (contract number)
    • DATE-SIGN (date contract signed)
    PHNUMBER PHNUMBER_REC PHNUMBER_COPY
    • OPERAT-NUM (operator number)
    • CONTR-NUM (contract number)
    • PHON-TYPE (phone type: S=smart phone, M=mobile)
    • PHON-NUM (phone number)
    PHONBILL PHONBILL_REC PHONBILL_COPY
    • PHON-TYPE (phone type: S=smart phone, M=mobile)
    • PHON-NUMBER (phone number)
    • TOT-CALL (total number of incoming and outgoing calls)
    • PERIOD (billing cycle)
  6. Continue to the next section.
Parent topic: Sequential Files Session


Loading Copybook Information into Your Mainframe Environment

Perform this procedure to map and load copybook information for your data store into the Knowledge Base. The information from the copybook will be mapped to your workspace.

Load copybook information into the Knowledge Base:

  1. 1. From Data Builder, open the Work with Jobs window by clicking Environment > Work with Jobs.
  2. 2. Click New.
  3. 3. In the List of Jobs, select the job BURLCPY (Load Copybook Information From External Interface).
  4. 4. Select the DATAEXPRES Machine ID you just created.
  5. 5. Specify DEMO as the name of your company code in the Company name box.
  6. Click Apply.
  7. 6. In the Sequential data store box, enter: URADAR.DEMOT.EXTCPY.
  8. 7. Click OK. The job appears in the List of Scheduled Jobs. The pause symbol indicates that the job is awaiting execution.
  9. 8. Continue to the next section.
Parent topic: Sequential Files Session


Loading Data Store Information into Your Mainframe Environment

Perform this procedure to map and load the definitions from your data store into the Knowledge Base. The information from your data store will be mapped to your workspace.

  1. From the Work with Jobs window, click New.
  2. In List of Jobs, select the job BURLFIL (Load Data Store Information From External Interface).
  3. Select the DATAEXPRES Machine ID you just created.
  4. Specify DEMO as the name of your company code in the Company name box.
  5. Click Apply.
  6. In the Sequential data store box, enter: URADAR.DEMOT.EXTSEQ.
  7. Click OK. The job appears in the List of Scheduled Jobs. The pause symbol indicates that the job is awaiting execution. Once the job has been submitted on the mainframe and has completed successfully, the job is listed with a green check mark.
  8. In your TSO environment, submit the BURLFIL and BURLCPY jobs by using the Submit Client Scheduled Job panel.
    Note: The Submit Client Scheduled Job panel is accessible by selecting option Submit Client Scheduled Job from the Data Express for z/OS - Main Menu panel. For more information, see the Submission Function in the Process Guide for z/OS.
  9. In the Work with Jobs area, click the Refresh button to verify that jobs completed successfully.
  10. Close the Work with Jobs window.
  11. In the [Work with Data Stores] window, expand the Machine ID node, and select DATAEXPRES, to view data stores associated with the DATAEXPRES machine ID:
  12. If you’re planning to go straight on to the next session, you can keep Data Builder open. Otherwise, click Exit and then click Yes.
Parent topic: Sequential Files Session


Data Masking

The Data Masking module is used to manage the privacy of data contained in databases of applications that are either developed internally or purchased by external suppliers. The aim is to protect sensitive data such as name, surname, or company name according to privacy laws and regulations.

This tutorial explains the Data Masking configuration procedure and describes the steps required to migrate the files in your application.

Note: Sessions in this tutorial should be done in order.

This tutorial walks you through the data classification and the data masking phases.

Begin this session only after you have set your workspace and loaded data store information into the Knowledge Base in tutorial Using Data Express and Data Inventory. You must already be logged in to Data Builder to begin.

The sample sessions are broken out into two parts: one for DB2 data stores and one for sequential files.

Note: Sessions in this tutorial must be done in order. If you choose to complete the tutorial for both DB2 and sequential data, then do all sessions. If you choose to complete the tutorial for only DB2 or sequential data, then follow the sessions only for that data type.
  • DB2 Data Stores Session
    Describes the DB2 data store session for data masking.
  • Sequential Files Session
    Descibes the sequential files session.
Parent topic: Data Express for z/OS Tutorials


DB2 Data Stores Session

In this session, you :

  • Associate supplied masking classes to data elements
  • Start the Data Masking module
  • Enable masking on the DB2 table CUSTOMER for NAME and SURNAME classes
  • Submit a JCL to mask customer first and last names
  • Create a masking class
  • Associate a masking class to data elements
  • Enable masking on the DB2 table for the CODCUS class
  • Submit a JCL to mask customer IDs
  • Associating Supplied Masking Classes to Data Elements
    Describes how to associate supplied masking classes to data elements.
  • Starting the Data Masking Module
    Describes how to start the data masking module.
  • Enabling Masking on the DB2 Table CUSTOMER
    Describes how to enable masking on the DB2 table CUSTOMER.
  • Submitting a JCL to Mask Customer First and Last Names
    Describes how to submit a JCL to mask customer first and last names.
  • Creating a Class
    Describes how to create a class.
  • Associating a Class to Data Elements
    Describes how to associate a class to data elements.
  • Enabling Masking on the DB2 Table CUSTOMER for the CODCUS Class
    Descibes how to enable masking on the DB2 table CUSTOMER for the CODCUS class.
  • Submitting a JCL to Mask Customer IDs
    Describes how to submit a JCL to mask customer IDs.
Parent topic: Data Masking


Associating Supplied Masking Classes to Data Elements

In this exercise you will be manually associating classes supplied with your installation with columns NAME and SURNAME in the table CUSTOMER:

  1. In the [Work with Data Stores] window, ensure that the Machine ID node is expanded, and that DATAEXPRES is selected:
  2. In the List of Data Stores, select the row for the DELGT.CUSTOMER data store.
  3. Right-click and then select Work with Data Elements of the Selected Data Store.

    If a message appears indicating that the default confirmation options have not been set, click Yes to set them. Click Apply, and then OK in the Options window.

    You can also access the Work with Data Elements window by clicking or by accessing it through Environment > Work with Data Elements . However, if you choose to access the Work with Data Elements window directly, you may want to specify information in the fields above the List of Data Elements and then click Apply Filter.

  4. Click the Classes - Assignment tab. This screen shows a filtered list based on Machine ID and Company Name.
  5. In the All Classes pane, expand the Privacy Super Class, and then select the class NAME.
  6. In the List of Data Elements, select the row for CUSTOMER.NAME. (That is, select the row for the data store CUSTOMER and the name NAME.)
    Note: If you are viewing all data stores, you could click the Data Store column to s sort data elements by data store.
  7. Click the Assign Class button to assign the NAME class to CUSTOMER.NAME. NAME now appears in the Assigned Class column for CUSTOMER.NAME.
  8. In the List of Data Elements, select the row for CUSTOMER.SURNAME. (That is, select the row for the data store CUSTOMER and the name SURNAME.)
  9. Click the Assign Class button to assign the SURNAME class to CUSTOMER.SURNAME. SURNAME now appears in the Assigned Class column for CUSTOMER.SURNAME.
  10. Click Close.
  11. Leave Data Builder running, and continue to the next section.
Parent topic: DB2 Data Stores Session


Starting the Data Masking Module

  1. Click Start > All Programs > Micro Focus Data Express 4.0 > Data Masking.
  2. Click OK.
  3. Select the mainframe Knowledge Base as the required database with which to launch the connection. In this example, the database is MVSDB2. By default, the highlighted item in the list of available databases is the last database you connected to using Data Masking. This becomes apparent after you click Available databases.
  4. Click OK.
  5. Log in using your mainframe user name and password.
  6. Click OK.
  7. Select the schema OWUR40SV, and click OK.

    The Main Window comprises four main areas:

    • Machine ID./Company grid
    • All Machine ID./Companies
    • Multiple Class Selection
    • Object Distribution

    The area containing the grid and the tabs Enabled and Disabled is referred to as the Machine ID./Company area or the Grid area. The grid lists files related to the selected class and to the selected machine ID and company. The names of the forms, their meaning and the included information are outlined below.

    The Enabled tab shows a list of all active files (enabling a file means that it is visible in the export functionality in order to be prepared for masking), and all files, which have one or more specific classes related to their data elements.

    The Object Distribution area displays a pie chart and bar chart representation of the file types for the classes selected in the Machine ID./Companies or Multiple Class Section area. The pie chart shows the percentage distribution of file types, while the bar chart shows the numerical distribution of file types. For both charts, both enabled and disabled files are included. Click the minimize button to minimize the Object Distribution area.

  8. Continue to the next section.
Parent topic: DB2 Data Stores Session
Parent topic: Sequential Files Session


Enabling Masking on the DB2 Table CUSTOMER

Data masking lets you to enable masking for individual data stores or all data stores, and to enable masking at the data-element level or class level. You can mask at the class level or at the data-element level. In this tutorial, we will be masking at the class level.

In this exercise you will activate masking for the data store CUSTOMER at the class (NAME and SURNAME) level.

To mask at the data-element level, double-click the desired data store from the Main Window, right-click the desired data element, and then click Data Changer Data Element Properties. Click Change, click Yes to activate the Changer option, and finally click Apply to activate your setting.

  1. In data masking, click the ALL Machine ID./Companies list.
  2. Expand the All Machine ID./Companies node.
  3. Double-click the DATAEXPRES machine ID.
  4. Select the row for the data store DELGT.CUSTOMER.
  5. Right-click and select Enable. The data store disappears from the list.
  6. Verify CUSTOMER is set as the Input Unload data store name and specify URADAR.DEMOT.MASK.new_library_name as the Output Unload data store name. For example: URADAR.DEMOT.MASK.DB2CUST.
  7. Click OK.
  8. Click the Enabled tab.
    Note: You can also choose to enable all data stores that have assigned classes with masking routines by choosing Tools > Enable All Data Stores.
  9. Select the data store DELGT.CUSTOMER, right-click, and then click Open Selected Data Store.
  10. Expand the row for the SURNAME node.
  11. Click the SURNAME class, right-click, and then click Enable.
  12. Expand the row for the NAME node.
  13. Click the NAME class, right-click, and then click Enable.
  14. Continue to the next section.
Parent topic: DB2 Data Stores Session


Submitting a JCL to Mask Customer First and Last Names

Use the Work with Jobs tool in data builder to submit the JCL that does the actual masking on the CUSTOMER table. Then, submit the job in the mainframe environment and verify your results.
  1. Data Builder, open the Work with Jobs window (click or access it through Environment > Work with Jobs) to schedule a new job.
  2. Click New.
  3. In the List of Jobs, select the job BDCCHGR (Data Store Data Changer).
  4. Select the DATAEXPRES Machine ID and the DEMO company name.
  5. In the Data store type list, click DB2.
  6. Click Apply.

    The job appears in the List of Scheduled Jobs. The pause symbol indicates that the job is awaiting execution. Once the job has completed successfully, the job is listed with a green check mark.

  7. In your TSO environment, submit the BDCCHGR job by using the Submit Client Scheduled Job panel.
  8. In the Work with Jobs area, click the refresh button to verify that the jobs completed successfully.
  9. Click OK or the button to close the Work with Jobs window.
  10. In your mainframe environment, notice that the values in the NAME and SURNAME columns for your input and output datasets are different.
  11. Continue to the next section.
Parent topic: DB2 Data Stores Session


Creating a Class

You can create new classes to be used for masking purposes, or you can use a predefined class. In this exercise you will create a masking class named CODCUS, which will enable us to mask the data element COD_CUS in the CUSTOMER table.

  1. In Data Builder, open the Work with Classes window (access it through Environment > Work with Classes)
  2. Click New.
  3. In the Name field, specify CODCUS as the name of the class you want to create.
  4. In the Full description field, specify COD_CUS MASK EXAMPLE as the description of the class. The description will be visible as a label when you assign the class to data elements within data stores.
  5. Specify UDCPIVC in the Data Masking Routine box.
    Note: This masking routine is provided during product installation. This is just an example routine for the purpose of creating a class. The masking logic for the routine is not in consideration for this demonstration.
    Note: You can mask at the class level or at the data-element level. In this tutorial, we will be masking at the class level.
  6. Select the Super Class Generated.
  7. Select the data type Alpha.
  8. Leave the remainder of the fields at their default values. Click Apply.
    Note: The class CODCUS is now listed in the List of Classes and is given the first free class ID.
  9. Click OK.
  10. Continue to the next section.
Parent topic: DB2 Data Stores Session


Associating a Class to Data Elements

In this exercise you will be manually associating the CODCUS class you just created with columns in the table CUSTOMER.

  1. In the [Work with Data Stores] window, expand the Machine ID node, select DATAEXPRES, and then select the row for the DELGT.CUSTOMER data store in the List of Data Stores.
  2. Right-click and then select Work with Data Elements of the Selected Data Store.
  3. Click the Classes - Assignment tab.
  4. In the All Classes pane, expand the Generated Super Class, and then select the class COD_CUS MASK EXAMPLE.
  5. In the List of Data Elements, select the row for CUSTOMER.COD_CUS. (That is, select the row for the data store CUSTOMER and the name COD_CUS.)
    Note: If you are viewing all data stores, you could click the Data Store column to s sort data elements by data store.
  6. Click the Assign Class button to assign the CODCUS class to CUSTOMER.COD_CUS. COD_CUS MASK EXAMPLE now appears in the Assigned Class column for CUSTOMER.COD_CUS.
  7. Click the Close button to close the Work with Data Elements window.
  8. Continue to the next section.
Parent topic: DB2 Data Stores Session


Enabling Masking on the DB2 Table CUSTOMER for the CODCUS Class

In this exercise you will activate masking for the data store CUSTOMER at the class (CODCUS) level.
  1. In Data Masking, click the ALL Machine ID./Companies list.
  2. Expand the All Machine ID./Companies node.
  3. Double-click the DATAEXPRES machine ID.
  4. On the Enabled tab, select the row for the data store DELGT.CUSTOMER.
  5. Right-click and select Data Changer Properties.
  6. Click Change.
  7. Verify CUSTOMER is set as the Input Unload data store name and specify URADAR.DEMOT.MASK.new_library_name as the Output Unload data store name. For example: URADAR.DEMOT.MASK.DB2CUST2.
    Note: Be sure that you change the output data store name in this exercise or you will overwrite the data from the previous masking exercise.
  8. Click OK.
  9. From the Enabled tab, select the data store DELGT.CUSTOMER, right-click and then click Open Selected Data Store.
  10. Expand the row for the node COD_CUS MASK EXAMPLE if it is not already expanded:
  11. Click the COD_CUS class, right-click, and then click Enable.
  12. Continue to the next section.
Parent topic: DB2 Data Stores Session


Submitting a JCL to Mask Customer IDs

Perform this procedure to map and load the definitions from your data store.
  1. 1. In Data Builder, open the Work with Jobs window (click Environment > Work with Jobs) to schedule a new job.
  2. Click New.
  3. In the List of Jobs, select the job BDCCHGR (Data Store Data Changer).
  4. Select the DATAEXPRES Machine ID and the DEMO company name.
  5. In the Data store type list, click DB2.
  6. Click Apply. The job appears in the List of Scheduled Jobs. The pause symbol indicates that the job is awaiting execution. Once the job has completed successfully, the job is listed with a green check mark.
  7. In your TSO environment, submit the BDCCHGR job by using the Submit Client Scheduled Job panel.
  8. In the Work with Jobs area, click the Refresh button to verify that the jobs completed successfully.
  9. Click OK or the button to close the Work with Jobs window.
  10. In your mainframe environment, notice that the values in the COD_CUS column for your input and output datasets are different.
  11. If you are planning to go straight on to the next tutorial to complete masking on sequential data, you can keep Data Builder open. Otherwise, click File > Exit, and then click Yes.
Parent topic: DB2 Data Stores Session


Sequential Files Session

In this session, you:

  • Associate supplied classes to data elements.
  • Start the Data Masking module.
  • Enable masking on the sequential file CUSTOMER for NAME and SURNAME classes.
  • Submit a JCL to mask customer first and last names.
Note: The sequential files session does not cover creating a masking class like the DB2 session does.
  • Associating Supplied Classes to Data Elements
    Describes how to associate supplied classes to data elements.
  • Starting the Data Masking Module
    Describes how to start the data masking module.
  • Enabling Masking on the Sequential File CUSTOMER and NAME and SURNAME classes
    Describes how to enable masking on the sequential file CUSTOMER and NAME and SURNAME classes.
  • Submitting a JCL to Mask Customer First and Last Names
    Describes how to submit a JCL to mask customer first and last names.
Parent topic: Data Masking


Associating Supplied Classes to Data Elements

In this exercise you will manually associate classes supplied with your installation with columns NAME and SURNAME in the table CUSTOMER.
  1. In the [Work with Data Stores] window, ensure that the Machine ID node is expanded, and that DATAEXPRES is selected:
  2. In the List of Data Stores, select the row for the URADAR.DEMOT.CUSTOMER data store.
  3. Right-click and then select Work with Data Elements of the Selected Data Store.
    Note: If a message appears indicating that the default confirmation options have not been set, click Yes to set them. Click Apply, and then OK in the Options window.
    Note: You can also access the Work with Data Elements window by clicking or by accessing it through Environment > Work with Data Elements. However, if you choose to access the Work with Data Elements window directly, you may want to specify information in the fields above the List of Data Elements and then click Apply Filter.
  4. Click the Classes - Assignment tab.
  5. In the All Classes pane, expand the Privacy Super Class, and then select the class NAME.
  6. In the List of Data Elements, select the row for CUSTOMER.NAME. (That is, select the row for the data store CUSTOMER and the name NAME.)
    Note: If you are viewing all data stores, you could click the Data Store column to sort data elements by data store.
  7. Click the Assign Class button to assign the NAME class to CUSTOMER.NAME. NAME now appears in the Assigned Class column for CUSTOMER.NAME.
  8. In the List of Data Elements, select the row for CUSTOMER.SURNAME. (That is, select the row for the data store CUSTOMER and the name SURNAME.)
  9. Click the Assign Class button to assign the SURNAME class to CUSTOMER.SURNAME. SURNAME now appears in the Assigned Class column for CUSTOMER.SURNAME.
  10. Click the Close button to close the Work with Data Elements window.
  11. Leave Data Builder running, and continue to the next section.
Parent topic: Sequential Files Session


Enabling Masking on the Sequential File CUSTOMER and NAME and SURNAME classes

Data Masking lets you enable masking for individual data stores or all data stores, and to enable masking at the data-element level or class level.

In this exercise you will be masking at the class level by activating masking for the data store CUSTOMER at the class (NAME and SURNAME) level.

Note: To mask at the data-element level, double-click the desired data store from the Main Window, right-click the desired data element, and then click Data Changer Data Element Properties. Click Change, click Yes to activate the Changer option, and finally click Apply to activate your setting.
  1. In Data Masking, click the ALL Machine ID./Companies list.
  2. Expand the All Machine ID./Companies node
  3. Double-click the DATAEXPRES machine ID.
  4. Select the row for the data store URADAR.DEMOT.CUSTOMER.
  5. Right-click and select Enable.
  6. Click the Enabled tab, and then right-click and select Data Changer Properties.
  7. Verify URADAR.DEMOT.CUSTOMER is set as the Input Unload data store name and specify URADAR.DEMOT.MASK.new_library_name as the Output Unload data store name. For example: URADAR.DEMOT.MASK.SEQCUST.
  8. Click OK.
  9. Click the Enabled tab.
  10. Select the data store URADAR.DEMOT.CUSTOMER, right-click, and then click Open Selected Data Store.
  11. Expand the row for the SURNAME node.
  12. Click the SURNAME class, right-click, and then click Enable.
  13. Expand the row for the NAME node.
  14. Click the NAME class, right-click, and then click Enable.
  15. Continue to the next section.
Parent topic: Sequential Files Session


Submitting a JCL to Mask Customer First and Last Names

Use the Work with Jobs tool in Data Builder to submit the JCL that does the actual masking on the CUSTOMER table. Then, submit the job in the mainframe environment and verify your results.
  1. In Data Builder, open the Work with Jobs window (click Environment > Work with Jobs) to schedule a new job.
  2. Click New.
  3. In the List of Jobs, select the job BDCCHGR (Data Store Data Changer).
  4. Select the DATAEXPRES Machine ID and the DEMO company name.
  5. In the Data store name box, type: *CUSTOMER*.
  6. Click Apply. The job appears in the List of Scheduled Jobs. The pause icon indicates that the job is awaiting execution. Once the job has completed successfully, the job is listed with a green check mark.
  7. In your TSO environment, submit the BDCCHGR job by using the Submit Client Scheduled Job panel.
  8. In the Work with Jobs area, click the refresh button to verify that the jobs completed successfully.
  9. Click OK to close the Work with Jobs window.
  10. In your mainframe environment, notice that the values in the NAME and SURNAME columns for your input and output datasets are different.
Parent topic: Sequential Files Session


Simple Data Subset Extraction

The Data Subset Extraction module is used to generate a reduced test environment. In this tutorial, you perform basic data subset extraction.

Note: Sessions in this tutorial should be done in order.

This tutorial walks you through the data classification and the data subset extraction phases. This session involves creating a simple subsetting class.

Begin this session only after you have set your workspace and loaded data store information into the Knowledge Base in tutorials Using Data Express and Data Inventory. You must already be logged in to Data Builder to begin.

The sample sessions are broken out into two parts: one for DB2 data stores and one for sequential files.

Note: Sessions in this tutorial must be done in order. If you choose to complete the tutorial for both DB2 and sequential data, then do all sessions. If you choose to complete the tutorial for only DB2 or sequential data, then follow the sessions only for that data type.
  • DB2 Data Stores Session
    Describes the DB2 data stores session.
  • Sequential Files Session
    Describes the sequential files session.
Parent topic: Data Express for z/OS Tutorials


DB2 Data Stores Session

In this session, you:

  • Associate a class to a data element.
  • Start the Data Subset Extraction module.
  • Create a group.
  • Create a simple method within that group.
  • Submit a JCL to subset customer names.
  • Associating a Class to a Data Element
    Describes how to associate a class to a data element.
  • Starting Data Subset Extraction
    Describes how to start data subset extraction.
  • Creating a Group
    Describes how to create a group.
  • Creating a Simple Method within a Group
    Describes how to create a simple method within a group.
  • Submitting a JCL to Subset Customer Names
    Describes how to submit a JCL to subset customer names.
Parent topic: Simple Data Subset Extraction


Associating a Class to a Data Element

In this exercise, you will be manually associating the NAME predefined class supplied with your installation with the column NAME in the table CUSTOMER.

Note: Similarly, if you performed the masking tutorial for DB2 data stores, then you already have the correct class associations for your data elements; you can skip the instructions in this section. If you have not performed the masking tutorial, then you cannot skip this section.
  1. In the [Work with Data Stores] window, ensure that the Machine ID node is expanded, and that DATAEXPRES is selected.
  2. In the List of Data Stores, select the row for the DELGT.CUSTOMER data store.
  3. Right-click and then select Work with Data Elements of the Selected Data Store.
    Note: If a message appears indicating that the default confirmation options have not been set, click Yes to set them. Click Apply, and then OK in the Options window.
    Note: You can also access the Work with Data Elements window by clicking or by accessing it through Environment > Work with Data Elements. However, if you choose to access the Work with Data Elements window directly, you may want to specify information in the fields above the List of Data Elements and then click Apply Filter.
  4. Click the Classes - Assignment tab.
    Note: This screen capture shows a filtered list based on Machine ID and Company Name.
  5. In the All Classes pane, expand the Privacy Super Class, and then select the class NAME.
  6. In the List of Data Elements, select the row for CUSTOMER.NAME. (That is, select the row for the data store CUSTOMER and the name NAME.)
    Note: If you are viewing all data stores, you could click the Data Store column to sort data elements by data store.
  7. Click the Assign Class button to assign the NAME class to CUSTOMER.NAME. NAME now appears in the Assigned Class column for CUSTOMER.NAME.
  8. Click the Close button to close the Work with Data Elements window.
  9. Leave Data Builder running, and continue to the next section.
Parent topic: DB2 Data Stores Session
Parent topic: Sequential Files Session


Starting Data Subset Extraction

Enter the context of your task here (optional).
  1. ClickStart > All Programs > Micro Focus Data Express 4.0 > Data Subset Extraction.
    Note: If you are using Data Express on Windows Vista and User Access Control is enabled, you must run Data Subset Extraction as Administrator.
  2. Select the mainframe Knowledge Base as the required database with which to launch the connection. In this example, the database is MVSDB2. By default, the highlighted item in the list of available databases is the last database you connected to using Data Masking. This becomes apparent after you click Available databases.
  3. Click OK.
  4. Log in using your mainframe username and password.
  5. Click OK.
  6. Select the schema OWUR40SV, and click OK.

    The Data Subset Extraction window comprises two main areas:

    • The node pane.
    • The grid area.
    The node pane contains a hierarchical structure that shows the grouping associated with each root-level node. The two root-level nodes are Groups and Creators.
  7. Continue to the next section.
Parent topic: DB2 Data Stores Session
Parent topic: Sequential Files Session


Creating a Group

The Work with Groups window lets you define, view, modify, and delete groups.

Groups can be used to logically separate the methods created for each workspace. Each method must belong to a group.

  1. Open the Work with Groups window (Environment > Work with Groups).
  2. Click New.
  3. In the Machine ID list, click DATAEXPRES.
  4. In the Company list, click DEMO.
  5. In the Group name box, specify SIMPLESUB.
  6. In the Group description box, type GROUP FOR SIMPLE SUBSET EXERCISE.
  7. Click Apply.
  8. Click OK.
  9. Continue to the next section.
Parent topic: DB2 Data Stores Session
Parent topic: Sequential Files Session


Creating a Simple Method within a Group

The New Method Wizard is used to create a new method, which is the extraction proposal containing the set of operations needed for the creation of the test environment. During method creation, all the information concerning the method is saved locally in a directory indicated by the user

Reduction can be obtained for data stores of a certain size and/or a particular type. In this case, you must be knowledgeable about the application to determine the appropriate selections. However, the Data Subset Extraction module can recognize and automatically create a first subset containing a series of files (called register table files) that are initially brought into the new test environment. Identifying register table files has the following advantages:

  • You can create a method to extract only the register table file subset, which is normally valid for the creation of any test environment independently of the selection criteria applied. Once this method has been saved and confirmed, it can be run whenever necessary.
  • You can create a second file subset, which filters the register file subset you created, to produce a file subset that is reduced even further.

To create a simple method:

  1. Launch the New Method Wizard (Environment > Create New Method).
  2. In the Machine ID list, click DATAEXPRES.
  3. In the Company list, click DEMO. Files belonging to different workspaces cannot be included.
  4. Click Next to go to the next screen.
    Note: If a group has not previously been defined in the Work with Groups area, it can be defined from this screen, by simply specifying its name and description in a specific field.
  5. In the Group name box, click SIMPLESUB.Text in the Group description box populates automatically.
  6. In the Method box, type SIMPLEDB2.
  7. In the Method description box, type: SIMPLE DB2 METHOD.
  8. Click Next.
  9. Select Import all non-register table Data Stores to include demographic files are registered in a dedicated elaboration step of the method.
  10. Click to clear the Import all register table Data Stores box.
  11. Click Next to go the next screen.
  12. Select NAME to be used as the primary extraction criteria.
  13. Click Next.

    The Data Subset Extraction module automatically extracts all the environment files containing the selected classes, entering them into a specific step. Once the method has been created, the remaining files (that is the files that are not sensitive to the selected classes) will initially be excluded from elaboration. During the Method Confirmation phase, they can be included in the method again. These files will then be integrally copied into the new environment generated by the method involved.

    The primary extraction criteria are listed in the Selection class field.

  14. For this tutorial, specifying a selection class in the Other Class is not necessary because you are creating a method for only one data store. You do not need another class. Note that both the DB2 and sequential data stores used throughout these tutorials both use the class NAME. Click Finish.
  15. Click Yes.

    The Data Subset Extraction module automatically extracts all the environment files containing the related classes and inserts them in a further elaboration step.

    The Create Method window is displayed:

  16. Click Start.
  17. Click OK.
  18. Click Confirm.

    During method confirmation, specified filters are applied to the classes needed to perform the extraction, and the method information is imported from a temporary file to the Knowledge Base.

    In this example, we will simply create a method based on the data stores that use the class NAME.

  19. In the Prefix box, type: URADAR.DEMOT.SUB.DB2CUST.
  20. Click to clear the Move excluded data stores into new step box.
  21. Click OK.
  22. In the Choose a Filter Type list, click FILTER BY VALUE LIST.
  23. In one of the boxes on the lower left side, type: Rick.
  24. Click OK to close the Set Filter to Selection Classes window.
  25. In the Unload Output Data Store Name box, type: URADAR.DEMOT.SUB.DB2CUST
  26. Click OK to confirm the method.
  27. In the Work with Method window, click Properties:
  28. Click Active.
  29. Click OK.
  30. Click OK.
  31. Continue to the next section.
Parent topic: DB2 Data Stores Session
Parent topic: Sequential Files Session


Submitting a JCL to Subset Customer Names

Use the Work with Jobs tool in Data Builder to submit the JCL that does the actual subsetting on the CUSTOMER table. Then, submit the job in the mainframe environment and verify your results.
  1. In Data Builder, open the Work with Jobs window (Environment > Work with Jobs) to schedule a new job.
  2. Click New.
  3. In the List of Jobs, select the job BTESBMR (test environment creation).
  4. Select the DATAEXPRES Machine ID and the DEMO company name.
  5. In the Data store name box, type: *CUSTOMER*.
  6. Click Apply.
  7. In the Group list, click SIMPLESUB.
  8. In the Method list, click SIMPLE.
  9. Click OK. The job appears in the List of Scheduled Jobs. The pause symbol indicates that the job is awaiting execution. Once the job has completed successfully, the job is listed with a green check mark.
  10. In your TSO environment, submit the BTESBMR job by using the Submit Client Scheduled Job panel.
  11. In the Work with Jobs area, click the Refresh button to verify that the jobs completed successfully.
  12. Click OK or the button to close the Work with Jobs window.
  13. In your mainframe environment, notice that only information for the value Rick in the NAME column for your output dataset is listed.
    Note: Because the Name class is associated to a masking routine, the data was masked as well as reduced.
  14. If you are planning to go straight on to the next tutorial to complete subsetting on sequential data, you can keep Data Builder open. Otherwise, either click File > Exit and then click Yes.
Parent topic: DB2 Data Stores Session
Parent topic: Sequential Files Session


Sequential Files Session

In this session, you:

  • Associate a class to a data element.
  • Start the Data Subset Extraction module.
  • Create a group.
  • Create a simple method within that group.
  • Submit a JCL to subset customer names.
  • Associating a Class to a Data Element
    Describes how to associate a class to a data element.
  • Starting Data Subset Extraction
    Describes how to start data subset extraction.
  • Creating a Group
    Describes how to create a group.
  • Creating a Simple Method within a Group
    Describes how to create a simple method within a group.
  • Submitting a JCL to Subset Customer Names
    Describes how to submit a JCL to subset customer names.
Parent topic: Simple Data Subset Extraction


Data Subset Extraction using Referential Integrity and Combined Data Elements

In this tutorial, you import class information using referential integrity for a DB2 data store. Then you create a method RIMETHOD to use a combined data element.

A combined data element is essentially a dummy data element that you create for ease of use when it comes to manipulating classes. Once a combined data element is created, you can perform operations against the data element like you would for other elements.

Note: Sessions in this tutorial should be done in order.

This tutorial walks you through the data classification and the data subsetting phases. In this session, subsetting is based on referential integrity rules.

Begin this session only after you have completed the DB2 Data Stores Session for the tutorial Simple Data Subset Extraction. You must already be logged in to Data Builder to begin.

In this session, you:

  • Import referential integrity classes.
  • Assign estimated classes.
  • Verify combined data element information.
  • Create a referential integrity method.
  • Edit method properties.
  • Submit a JCL to create the test environment.
  • Creating the HSURDRIA table
    Describes how to create the HSURDRIA table.
  • Importing Referential Integrity Classes
    Descibes how to import referential integrity classes.
  • Assigning Estimated Classes
    Describes how to assign estimated classes.
  • Verifying Combined Data Element Information
    Describes how to verify combined data element information.
  • Creating a Referential Integrity Method
    Describes how to create a referential integrity method.
  • Editing Method Properties
    Describes how to edit method properties.
  • Submitting a JCL to Create the Test Environment
    Describes how to submit a JCL to create the test environment.
Parent topic: Data Express for z/OS Tutorials


Creating the HSURDRIA table

The ability to import referential integrity class is based on the ACTIVATION field in the HSURDRIA table located in the Data Express knowledge base. During the z/OS Host installation this table is not created in the knowledge base. It must be created manually in case the import type Referential Integrity in the Import Class window will be used. This way of handling automatic referential integrity has been chosen in order to prevent uncontrolled access to z-OS knowledge base from user interface, because the referential integrity process is a time consuming one and it can affect the z-OS performances.

  1. Ensure administrative privileges for the DB2 userid.
  2. Create the table HSURDRIA using the DDL below:
    CREATE TABLESPACE      HSURDRIA
       IN               &URDBASE
       USING STOGROUP   &URSGROUP
       PRIQTY         12
       SECQTY         12
       FREEPAGE       0
       PCTFREE        0
       BUFFERPOOL        &URBPTS
       SEGSIZE        64
       COMPRESS          NO;

CREATE TABLE &UROWNER.HSURDRIA
       FREEPAGE    0
       PCTFREE     0
       BUFFERPOOL    &URBPTS
       SEGSIZE     64
       COMPRESS      NO;

CREATE TABLE  &UROWNER.HSURDRIA
   (ACTIVATION CHAR(1) NOT NULL WITH DEFAULT)
 IN  &URDBASE..HSURDRIA;

CREATE UNIQUE INDEX &UROWNER.HSURDRIAX1 ON &UROWNER..HSURDRIA
 (ACTIVATION)
        USING STOGROUP &URSGROUP
        PRIQTY 12
        SECQTY 12
        CLUSTER
        BUFFERPOOL  &URBPIX;
    &URDBASE
    Name of the DB2 database where the product tables are created.
    &URSGROUP
    Name of the DB2 storage group.
    &UROWNER
    Name of the DB2 creator that is used in order to create all the product tables.
    &URBPTS
    Name of the buffer pool containing the data of the product tables.
    &URBPIX
    Name of the buffer pool containing the indexes of the product tables.
  3. To activate the import type Referential Integrity, insert a value Y in the field ACTIVATION using the command: INSERT INTO &UROWNER.HSURDRIA (ACTIVATION) VALUES ('Y')
  4. After restarting Data Builder you should see import type Referential Integrity active.
  5. To deactivate the import type Referential Integrity perform the command below and then restart the Data Builder: DELETE FROM &UROWNER.HSURDRIA WHERE ACTIVATION = 'Y'
Parent topic: Data Subset Extraction using Referential Integrity and Combined Data Elements


Importing Referential Integrity Classes

In this exercise you import referential integrity classes by running the BURDDUR (Import Classification From Referential Integrity) job, which creates a referential integrity class and combined data elements, and creates an estimated class assignment for those data elements.

Note: You can also use the Import Class window to import referential integrity classes. However, this mechanism is designed to be used with the New Method Wizard to create a simple method with referential integrity that supports two-level relationships between data stores.
  1. In Data Builder, open the Work with Jobs window (click or access it through Environment > Work with Jobs).
  2. Click New.
  3. In List of Jobs, select the job BURDDUR (Import Classification From Referential Integrity).
  4. Select the DATAEXPRES Machine ID.
  5. Specify DEMO as the name of your company code in the Company name box.
  6. Click Apply.
  7. In the Referential Integrity Interface box, type: URADAR.DEMOT.REFINTEG 8. Click OK.

    The job appears in the List of Scheduled Jobs. The pause symbol indicates that the job is awaiting execution. Once the job has completed successfully, the job is listed with a green check mark.

  8. In your TSO environment, submit the BURDDUR job by using the Submit Client Scheduled Job panel.
  9. In the Work with Jobs area, click the Refresh button to verify that the jobs completed successfully.
  10. Close the Work with Jobs window.
  11. Continue to the next section.
Parent topic: Data Subset Extraction using Referential Integrity and Combined Data Elements


Assigning Estimated Classes

In this exercise, you assign estimated classes to the combined data elements created by the BURDDUR job and to the data elements that compose those combined data elements.

  1. Open the Work with Data Elements window (Environment > Work with Data Elements).
    Note: You can also access the Work with Data Elements window from the main window by selecting the file name, right-clicking to view the context menu, and then by clicking Work with Data Elements of the Selected Data Store.
  2. From the Select Data Elements tab, specify DATAEXPRES as the Machine ID and DEMO as the Company name.
  3. Click the Data Store Attributes tab.
  4. In the DB2 Owner, DL/I DBD list, click DELGT.
  5. Click Apply Filter to further narrow down the list of data elements displayed.
  6. In the List of Data Elements (Filtered) table, view the column Estimated Class.

    The BURDDUR job you ran has created four combined data elements and has automatically estimated class assignments. These assignments are estimated for the newly created combined data elements and for the data elements that compose these combined data elements. (Likewise, if a data element that composes a combined data element already has an assigned class, the data element is not associated with an estimated class.)

    Note: When you performed the Data Masking exercise, the class CODCUS (displayed by its class description COD_CUS MASK EXAMPLE) was assigned to the COD_CUS data element in the CUSTOMER data store. Then, when you ran the BURDDUR job as part of this exercise, the class CODUS became an estimated class assignment for the COD_CUS data element in the ACCOUNT data store.
  7. In the List of Data Elements (Filtered), view the newly assigned estimated classes created for the following data stores:
    ACCOUNT and CCARD data stores
    • CLA0449 - created for the data element OFF_NUM.
    • CLA0450 - created for the data element ACC_NUM.
    • CLA0451 - created for the combined data element, which is comprised of both OFF_NUM and ACC_NUM data elements.
    CCARD and OPERAT data stores
    • CLA0452 - created for the data element CARD_TYPE
    • CLA0453 - created for the data element CARD_NUM
    • CLA0454 - created for the combined data element, which is comprised of both CARD_TYPE and CARD_NUM
    Note: The actual names of your estimated classes will differ as they are system-generated.
  8. Click the Estimated Classes - Confirm tab
  9. In the List of Data Elements (Filtered), select each row that has an estimated class but not an assigned class.
    Note: Not all selected rows are displayed.
  10. Click Confirm Class.

    The Estimated Class names now appear in the Assigned Class column.

  11. Close the Work with Data Elements window to return to the Work with Data Stores window.
  12. Continue to the next section.
Parent topic: Data Subset Extraction using Referential Integrity and Combined Data Elements


Verifying Combined Data Element Information

In this exercise, you will view and edit combined data element properties.
  1. From the Work with Data Stores window, select a data store that you know has a combined data element. (For this exercise, you can select DELGT. ACCOUNT, DELGT.CCARD, or DELGT.OPERAT.)
  2. Right-click your selection, and choose Open Selected Data Store. For this demonstration, the data store DELGT.ACCOUNT was chosen.
  3. In the List of Data Elements, select the row for the combined data element, right-click, and then click Work with Combined Data Element.
  4. View properties of the system-generated combined data element. Change the Combined Data Element Attributes if desired.
  5. Click Apply or Close as appropriate.
  6. Close the Work With Data Store window.
  7. Continue to the next section.
Parent topic: Data Subset Extraction using Referential Integrity and Combined Data Elements


Creating a Referential Integrity Method

In this exercise you will create a referential integrity method by running the BTEDDUR (Import Method From Referential Integrity) job.

Note:

  • After class assignments have been established, the BTEDDUR job is used to create a complex method with referential integrity that supports multiple-level relationships between data stores.

  • You can also use the New Method Wizard to create a simple method with referential integrity, but the wizard is limited in that it only supports two-level relationships between data stores.

  • The method created by the BTEDDUR is automatically activated during job execution. Therefore, you do not need to activate the method in the Method Properties window.

  1. 1. In Data Builder, open the Work with Jobs window (Environment > Work with Jobs).
  2. Click New.
  3. In List of Jobs, select the job BTEDDUR (Import Method From Referential Integrity).
  4. Select the DATAEXPRES Machine ID.
  5. Specify DEMO as the name of your company code in the Company name box.
  6. Click Apply.
  7. In the Referential Integrity Interface box, type: URADAR.DEMOT.REFINTEG.
  8. In the Group list, click SIMPLESUB.
    Note: You created the SIMPLESUB group during the Simple Subset Extraction exercise for DB2 data stores.
  9. In the Method field, type: RIMETHOD.
    Note: The method RIMETHOD is created when the job runs.
  10. In the Method text box, type: METHOD FOR RI AND COMBINED DATA ELEMENT.
  11. In the Class list, click COD_CUS MASK EXAMPLE.
    Note: You created the class COD_CUS MASK EXAMPLE during the Data Masking exercise.
  12. Click OK.
Parent topic: Data Subset Extraction using Referential Integrity and Combined Data Elements


Editing Method Properties

In this exercise, you will edit the properties of the referential integrity method you just created.
  1. Launch Data Subset Extraction.
  2. In the left pane, expand the Group list, expand the SIMPLESUB list, and then select Methods.
  3. Select the row for the method RIMETHOD, right-click, and then choose Work with selected Method.

    The table information for all data stores has been exported because all data stores are related.

    Notice that the data stores DELGT.CUSTOMER and DELGT.ACCOUNT are elaborated in the step, and that the output class for the DELGT.ACCOUNT data store is the input class for DELGT.CCARD in the subsequent step. Likewise, the output class DELGT.CCARD is the input class for DELGT.OPERAT in the subsequent step.

    Elaborating the data stores in this way preserves referential integrity and shows that the DELGT.CUSTOMER data stores is indirectly related to the DELGT.OPERAT data store.

    The following diagram further demonstrations the data store relations:

    Data Store Model
    Note: The dashed line represents a relationship that exists between data stores because of the combined field.
  4. In the Data Stores Elaboration list, select a row for a DELGT.DataStoreName data store.

  5. Right-click and select Elaboration Properties
  6. In the Unload Output Data Store Properties section, in the Data store name box, type: URADAR.DEMOT.SUB2.DataStoreName. For example, type: URADAR.DEMOT.SUB2.CUSTOMER.
  7. Click OK.
  8. Repeat steps 4-7 for each DELGT.DataStoreName data store.
  9. In the Data Stores Elaboration list of the Work with Method window, select the row for the DELGT.CUSTOMER data store.
  10. Right-click, and then select Elaboration > Selection class / Filter properties.
  11. Choose the filter type FILTER BY LIST VALUE.
  12. In one of the boxes in the lower-left corner, type the following CUSTOMER CODE: 0000000001
    Note: You can verify the customer code in your z/OS DB2 window. However, this customer code is specified in an INSERT statement in the CREATBT.TXT file used to create your DB2 tables.
  13. Click OK to close the Set Filter to Selection Classes window.
    Note: The filter only needs to be specified for the CUSTOMER data store because the BTEDDUR job establishes the relations between all of the other data stores.
  14. Exit Data Subset Extraction by clicking File > Exit.
  15. Continue to the next section.
Parent topic: Data Subset Extraction using Referential Integrity and Combined Data Elements


Submitting a JCL to Create the Test Environment

Use the Work with Jobs tool in Data Builder to submit the JCL that does the actual subsetting on the CUSTOMER, ACCOUNT, CCARD, and OPERAT data stores. Then, submit the job in the mainframe environment and verify your results.
  1. 1. In Data Builder, open the Work with Jobs window (Environment > Work with Jobs) to schedule a new job.
  2. Click New.
  3. In the List of Jobs, select the job BTESBMR (Test Environment Creation).
  4. Select the DATAEXPRES Machine ID and the DEMO company name.
  5. Click Apply.
  6. In the Group list, click SIMPLESUB.
  7. In the Method list, click RIMETHOD.
  8. Click OK. The job appears in the List of Scheduled Jobs. The Pause symbol indicates that the job is awaiting execution. Once the job has completed successfully, the job is listed with a green check mark.
  9. In your TSO environment, submit the BTESBMR job by using the Submit Client Scheduled Job panel.
  10. In the Work with Jobs area, click Refresh to verify that the jobs completed successfully.
  11. Click OK.
  12. In your mainframe environment, verify the subset extraction results by viewing the sequential files created by the BTESBMR job:
    • URADAR.DEMOT.SUB2.CUSTOMER
    • URADAR.DEMOT.SUB2.ACCOUNT
    • URADAR.DEMOT.SUB2.CCARD
    • URADAR.DEMOT.SUB2.OPERAT
    Note: Because the COD_CUS MASK EXAMPLE class is associated to a masking routine, the data was masked as well as reduced.
  13. Exit Data Builder.
Parent topic: Data Subset Extraction using Referential Integrity and Combined Data Elements


Data Element Sampling

This tutorial walks you through the data sampling phases. In this tutorial, you perform sampling on sequential data stores to obtain information about the distribution of data. Then you use that distribution as a way to assign classes to other data elements with similar data distributions.

Note: Sessions in this tutorial should be done in order.

Complete this session only after you have completed the Configuring your z/OS Data Express Environment, Data Masking, and the Simple Data Subset Extraction tutorials.

You must already be logged in to Data Builder to begin. In this session, you:

  • Submit a JCL to perform sampling
  • Verify results
  • Create a sampling class
  • Associate a fingerprint to a class
  • Associate a class to data elements
  • Submitting a JCL to Perform Sampling
    Describes how to submit a JCL to perform sampling.
  • Verifying Results
    Describes how to verify results.
  • Creating a Sampling Class
    Describes how to create a sampling class.
  • Associating a Fingerprint to a Class
    Describes how to associate a fingerprint to a class.
  • Associating a Class to Data Elements
    Describes how to associate a class to data elements.
Parent topic: Data Express for z/OS Tutorials


Submitting a JCL to Perform Sampling

Use the Work with Jobs tool in Data Builder to submit the JCL that performs the actual sampling.
  1. From Data Builder, in the List of Data Stores in the Work with Data Stores window, expand the DATA EXPRES Machine ID and select the DEMO company.
  2. Select a data store with the Machine ID DATA EXPRES and the company DEMO, right-click, and then click Data Store Properties.

    Note that the Classification number is 1. By default all data stores are assigned a Classification number of 1, which means that sampling is enabled. Classification numbers are used to restrict the data stores that get sampled and to and to provide further granularity in restricting which data stores get sampled as based on user-defined criteria. When sampling the data stores within your work environment, all enabled data stores with Classification number values of 1 will be sampled. To disable sampling, change the Classification number to 0 in the Properties – File window or right-click the row for the data store in the Work with Data Stores and click Disable Sampling.

    The data store remains enabled as long as the Classification number is not 0. However, just because sampling is enabled does not mean that the data store is a candidate for sampling as this is controlled by the Classification number specified in the Secondary Options for the job that submits the JCL. If the Classification number set for the data store is less than or equal to the Classification number specified in the Distributed Sampler, the data store will be sampled. Likewise, if the data store Classification number is greater than the Classification number in the Distributed Sampler, the data store will not be sampled.

    Tip: You can assign classification numbers to your data stores based on how often you want to sample data. For example, 1 could represent daily sampling, 2 could represent weekly sampling, and 3 could represent monthly sampling.
  3. Click OK.
  4. Open the Work with Jobs window (Environment > Work with Jobs) to schedule a new job.
  5. Click New.
  6. In the List of Jobs, select the job BURSAMR (Data Store Data Element Sampling).
  7. Select the DATAEXPRES Machine ID and the DEMO company name.
  8. Click Apply.
  9. Retain all default sampling values. For this exercise, all data stores should be sampling candidates and all sampling options should be utilized.

    The following list describes the sampling Options:

    Compressed sampling

    Produces the data element fingerprint. The fingerprint graphically shows the distribution of values within a given range for the sampled data element. The fingerprints for numeric and alphanumeric data elements differ in that the fingerprint for an alphanumeric data element shows the distribution of values based on the first character and provides additional information.

    Standard sampling

    Displays information for each data element value including the number of times each value occurred and the percentage that value represents in the total population.

    Min/Max calculation

    Displays the minimum and maximum values for the data element.

  10. Click Advanced.
  11. Retain all default values.
    Note: The Data Element Size of 0 (zero) means to include all sizes in the sampling results and does not actually reflect the value of 0 when Ignore Special Values Zero / Space is checked. Likewise, the default 0 values in Max. And Min. Recalculation Additional Options mean to consider all values to be in-range and does not reflect the value of 0.
  12. Click OK.
  13. In your TSO environment, submit the BURSAMR job by using the Submit Client Scheduled Job panel.
  14. In the Work with Jobs area, click Refresh to verify that the job completed successfully.
  15. Close the Work with Jobs window.
  16. In your mainframe environment, verify the sampling results.
  17. Continue to the next section.
Parent topic: Data Element Sampling


Verifying Results

Once you have executed sampling, you can view the results.

When compressed sampling is performed, a fingerprint is created. The fingerprint is a unique graphical representation of the distribution of data for the sampled data element. When the results for compressed sampling are loaded, the fingerprint is created.

  1. In the List of Data Stores in the Work with Data Stores window, select the row for the URADAR.DEMOT.TableName data store.
  2. Click the Show Synthetic Data Elements button. The Data Store Data Elements list in the left pane shows the data elements for the data store you selected.
  3. In the Data Store Data Elements list, select the data element OPERAT-NUM.

    You can also select the image for the data element in the Data Element Samples grid. To ensure that you have selected the correct one, the related data element name is highlighted in the Data Store Data Elements list.

    Sampling results for the numeric data element OPERAT-NUM are displayed.

  4. Review the data element fingerprint in the Zoomed Data Element Sample. The fingerprint is the result of performing compressed sampling. Notice the following:
    • The numbers on the vertical y-axes represent the ranges of values based on the decimal place, where each number represents one decimal place to the left of the decimal point. For instance, 0 represents the ones place (0-9), 1 represents the tens place (10-99), 2 represents the hundreds place (100-999), and so on.
    • The numbers on the horizontal x-axes represent a specific range based on the y-coordinate corresponding to the first digit. For instance, 1 represents the value 1 for the y-coordinate 0, while 1 represents values 10-19 for the y-coordinate 1.
    • The values in each cell indicate the number of times a value in the data element falls within the specified range.
    Note: The fingerprint can also be viewed as a bar graph by clicking Graph.
  5. Review the items in the list Sample Analysis of the Selected Data Element. This list shows the actual data element values when standard sampling was performed.
    Note: Only the first 1500 distinct values and ranges display. They are sorted either numerically or alphanumerically based on the data element type.
  6. In the In the Data Store Data Elements list, select the data element COD-CUS. Sampling results for alphanumeric data element COD-CUS are displayed.
  7. Review the data element fingerprint in the Zoomed Data Element Sample. The fingerprint is the result of performing compressed sampling. Notice that the fingerprint for an alphanumeric data element contains four distinct sections:
    Section Type Description Example

    Character Description

    A range of characters in alphabetical order is provided to show how many data elements begin with that character.

     Spa A  B  C  D  E  F  G  H  I
  J  K  L  M  N  O  P  Q  R  S
  T  U  V  W  X  Y  Z        *

    Number Distribution

    A range of numbers is provided to show how many data elements begin with the indicated number. In the provided example, no data elements begin with numbers.

      0  1  2  3  4  5  6  7  8  9
  4

    Type Summary

    An alphanumeric data element can actually be a numeric-only data element or alphanumeric data element. The type summary provides a count of how many data elements fall into either category. In the provided example, all data elements are alphanumeric in type.

      Num Only          Alpha Only
  4

    Field Length

    The numbers on the vertical y-axes represent the ranges of values specified. The numbers on the horizontal x-axes represent the position of the actual number in that range. For instance, 4 in the provided example means that there are 4 values with a length equal to 10.

             Field Len.   Val.
         0  1  2  3  4  5  6  7  8  9

 0 -  9
10 - 19  4
20  -->
    Note: The fingerprint can also be viewed as a bar graph by clicking Graph.
  8. Close the Show Synthetic Data Element Contents window to return to Work with Data Stores.
  9. Open the Work with Data Elements window (Environment > Work with Data Elements).
  10. In the Machine ID list, click DATA EXPRES.
  11. In the Company name list, click DEMO.
  12. Click Apply Filter.
  13. Regard the Minimum Value and Maximum Value items in the List of Data Elements, which are the result of performing min/max calculation sampling.
  14. Leave the Work with Data Elements window open, and continue to the next section.
Parent topic: Data Element Sampling


Creating a Sampling Class

You can create a new class to be used for sampling purposes, or you can use a predefined class. In this exercise you will create a sampling class named SAMPCUS.

  1. 1. Open the Work with Classes window (click or access it through Environment > Work with Classes).
  2. Click New.
  3. In the Name box, specify SAMPCUS as the name of the class you want to create.
  4. In the Full description field, specify SAMPLING CUSTOMER VALUES as the description of the class.
    Note: The description will be visible as a label when you assign the class to data elements within data stores.
  5. Select the Super Class Generated.
  6. Check each data type box.
  7. Leave the remainder of the fields at their default values. Click Apply. The class SAMPCUS is now listed in the List of Classes.
  8. Click OK to return to the Work with Data Elements window.
  9. Click Refresh.
  10. Continue to the next section.
Parent topic: Data Element Sampling


Associating a Fingerprint to a Class

In Data Express, you can correlate data elements if their value distributions are similar. To do this, you must first associate a fingerprint, which represents the desired distribution, to a class.

This fingerprint becomes the prototype that is used determine the class assignment for other data elements. If the prototype and the fingerprint for a data element are deemed similar based on a calculated confidence value, the data element is also associated with the prototype class.

  1. Click the Classes – Assignments tab.
  2. In the All Classes pane, expand the Generated Super Class, and select the class description SAMPLING CUSTOMER VALUES.
  3. In the List of Data Elements, select the row for the data element COD-CUS in the data store URADAR.DEMOT.PHCONTRA.
  4. Check the Selected Data Element Attributes check box
  5. Drag the fingerprint that now appears beneath the text Selected Data Element Attributes, to the first Class Samples cell:

    By highlighting the SAMPCUS class description in the All Classes pane and then by dragging the fingerprint for the COD-CUS data element to the Class Samples cell, the SAMPCUS class is then assigned to the fingerprint. This fingerprint becomes the prototype. This action also associates the SAMPCUS class to the COD-CUS data element.

  6. Continue to the next section.
Parent topic: Data Element Sampling


Associating a Class to Data Elements

In this exercise you are associating the class SAMPCUS (which is associated to your sampling prototype) to other data elements with sampling data distributions similar to the prototype fingerprint.

The level of similarity between distributions depends on the thresholds you set when submitting the BURCLAR job. If the prototype and the fingerprint for a data element are deemed similar based on a calculated confidence value, the data element is also associated with the prototype class.

  1. 1. Open the Work with Jobs window (Environment > Work with Jobs) to schedule a new job.
  2. Click New.
  3. In the List of Jobs, select the job BURCLAR (Class Data Element Assignment).
  4. Select the DATAEXPRES Machine ID and the DEMO company name.
  5. Click Apply. When comparing the fingerprint for a data element to the prototype fingerprint, internal program confidence values are calculated to provide measurements that illustrates the similarities between the data distributions. The internal confidence values are used as input for the threshold formulas.
  6. In the Select a class list, click SAMPLING CUSTOMER VALUES.
  7. Click Add. The class description now appears in the table.
  8. In the Rules section, uncheck By Attribute if it is checked.
  9. Set the Threshold 1 value to 50. The value for Threshold 1 represents a percentage where the similar areas in the two fingerprints are weighted more heavily than the dissimilar areas.
  10. Let the default value of 0 for Threshold 2. The value for Threshold 2 represents a percentage where all areas (both similar and dissimilar) in the two fingerprints are given equal weight.
  11. Keep the logical operator set to AND.
  12. Click OK.
  13. In your TSO environment, submit the BURCLAR job by using the Submit Client Scheduled Job panel.
  14. In the Work with Jobs area, click refresh to verify that the job completed successfully.
  15. Close the Work with Jobs area.
  16. In your mainframe environment, verify the class assignment.
  17. Return to the Work with Data Elements window and refresh the list of data elements.
  18. Select the Select Data Elements tab if it is not already selected.
  19. Click the Data Store Attributes tab.
  20. Delete any text in the Data store Name field.
  21. Click Apply Filter.
  22. Click the Estimated Classes - Confirm tab.
  23. In the List of Data Elements (Filtered), select the row that has an estimated class but not an assigned class.
  24. Click Confirm Class. The Estimated Class names now appear in the Assigned Class column.
  25. Close the Work with Data Elements window and return to the Work with Data Stores window. A sampling class has successfully been assigned to data elements with similar distributions to the prototype.
  26. Close Data Builder.
Parent topic: Data Element Sampling


Managing XML and Large-object Columns

Describes how to manage masking and subsetting of XML data stored in XML or LOB/BLOB/CLOB columns of z/OS DB2 tables, supporting both DSNTIAUL and UNLOAD formats.
  • Load and Classification
  • Masking
Parent topic: Getting Started with z/OS Data Stores


Load and Classification

Structure load of tables containing XML data, both in XML and in LOB/BLOB/CLOB columns, is performed via direct access DB2 load from the Work with Process IDs window, using the same load process as is used for tables that do not have these types of columns.

Work with Process IDs

XML and LOB/BLOB/CLOB columns are recorded in Data Express as VARCHAR(56), which is required to define a process identifier. In particular, be careful to always set the Access type for XML FILES correctly based on UNLOAD and DSNTIAUL keywords.

For example, the screenshot above cites Unload Data Store Access (UNLOAD), which is to be used for UNLOAD formats. If you are using the DSNTIAUL format, use Unload Data Store Access (DSNTIAUL) instead.

Class definition and assignment are the same as for normal columns. For all XML or LOB/BLOB/CLOB columns that are not being masked, we recommend that you define a special class to assign them to, and to assign them the UDCXML0 "dummy" masking routine.

Note: The information shown here is used only for masking and subsetting. It is not used for loading and cataloging.
Parent topic: Managing XML and Large-object Columns


Masking

You can use either the UNLOAD or the DSNTIAUL format for masking.

UNLOAD Format

Masking using the UNLOAD format requires a sequential unload for each table, and a PDS for each XML/LOB/BLOB/CLOB column. The following JCL provides an example of the required code:

//LOADLOB JOB KRM,MSGCLASS=A,CLASS=1,NOTIFY=&SYSUID                     JOB07656
//* *******************************************
//* TABLE OWURSVIL.BIG01
//* (PDFBLO BLOB (27994))
//* (TXTCLO CLOB (16094) )
//* (PDFBLO BLOB (16032) )
//* UNIT(SYSDA) SPACE ((20,2) MB)
//* (PDFBLO VARCHAR(54) BLOBF TSYSLOB)
//* ****************************************
//STEP1 EXEC DSNUPROC,UTPROC=,SYSTEM=HADB,LIB=DSNA10.SDSNLOAD
//SYSPRINT DD SYSOUT=*
//SYSIN DD *
  TEMPLATE TSYSPUN
     DSN('URADAR.&SS..&DB..PUNCH.CREDB1')
     DISP(MOD,CATLG,CATLG)
  TEMPLATE TSYSREC
     DSN('URADAR.&SS..&DB..SYSREC.CREDB1')
     DISP(MOD,CATLG,CATLG)
  TEMPLATE TSYSLOB
     DSN('URADAR.&SS..&DB..PDS.CREDB1')
     DISP(MOD,CATLG,CATLG)
     DSNTYPE(PDS)
  UNLOAD DATA FROM TABLE OWURSVIL.CREDB1 HEADER NONE
   (A     , B                  ,C      VARCHAR(54) CLOBF TSYSLOB,
    D)
  UNLDDN(TSYSREC) PUNCHDDN(TSYSPUN)
/*
//

In particular, use must include the HEADER NONE clause, and specify VARCHAR(54) for each XML/LOB/CLOB/BLOB column.

Before creating the method, be sure that you have created an output PDS via the masking routine, and that you have generated a process ID from Data Builder. When you create the method, include the input and output data set names.

DSNTIAUL Format

Masking using the DSNTIAUL format requires a sequential unload for each table. The following JCL provides an example of the required code:

//DB2UNLO  JOB (00001),URADAR,MSGCLASS=1,CLASS=1,NOTIFY=&SYSUID
//STEP0100 EXEC PGM=IKJEFT01,DYNAMNBR=20
//SYSTSPRT DD SYSOUT=*
//SYSTSIN  DD *
 DSN SYSTEM(HADB)
 RUN  PROGRAM(DSNTIAUL) PLAN(DSNTIB10)                -
       LIB('DSNA10.DB2HAL.RUNLIB.LOAD')
//SYSPRINT DD SYSOUT=*
//SYSUDUMP DD SYSOUT=*
//SYSREC00 DD DSN=URADAR.SV41.CREDB2.NEW,
//            DISP=(NEW,CATLG,DELETE),
//            SPACE=(TRK,(24,24),RLSE)
//SYSPUNCH DD DSN=URADAR.UTILITY.SYSPUNCH(CREDB2N),DISP=SHR
//SYSIN    DD *
                   OWURSVIL.CREDB2

Before creating the method, be sure that you have generated a process ID from Data Builder. When you create the method, include the input and output data set names.

  • XML Tag Identifiers
  • Provided Templates
Parent topic: Managing XML and Large-object Columns


XML Tag Identifiers

To pass data to a data masking routine from XML, you need to identify each data masking object in your XML code. A data masking object is one or more XML fields that each contain functionally equivalent data to be masked. You assign a tag identifier to each data masking object, and pass the identifier to the data masking routine as a literal string. This requires that you assemble your XML tags such that each data masking object can be uniquely identified by its elementary tag alone, or by qualification using its elementary and parental tags. The following examples illustrate this point:

Elementary tag identifier:

<Company>
   <Nm>Yoga Life Spa</Nm>
</Company>

<Client>
   <Nm>Hope Lesley Stressing</Nm>
</Client>

In this example, the <Nm> tag contains either a company name or a client name. Each <Nm> elementary tag instance is a child of a different parent – <Company> to identify a company, and <Client> to identify a client. Assuming company and client data are functional equivalents, the tag identifier used to represent this data masking object and that is passed to the data masking routine is just the elementary tag name, Nm, without further qualification.

Hierarchical tag identifier:

<OrgId>
   <IBAN>
      <Id>IT7788990010000004245520999</Id>
   </IBAN>
</OrgId>

<OrgId>
   <TaxCode>
      <Id>XYZABC40S18L424B</Id>
   </TaxCode>
</OrgId>

In this example, the <Id> tag is used to provide either the IBAN (International Bank Account Number) or the tax code associated with an organization. These are not functional equivalents, and therefore represent two different data masking objects. However, each is contained in an elementary <Id> tag; therefore, the tag identifier for each must be further qualified by including both the parent and the child tag. To do this, you specify the parent tag and its child tag in sequence, separated with a comma. For example: IBAN,Id and TaxCode,Id respectively.

Each tag identifier must include as many levels in the tag hierarchy as are required to uniquely identify the elementary tag that contains the data. For example, to uniquely identify each <Id> tag in the following code requires four levels of qualification: IBAN,Country,CountryCode,Id and TaxCode,Country,CountryCode,Id respectively.

<OrgId>
   <IBAN>
      <Country>
         <CountryCode>
            <Id>IT7788990010000004245520999</Id>
         </CountryCode>
      </Country>
   </IBAN>
</OrgId>

<OrgId>
   <TaxCode>
      <Country>
         <CountryCode>
            <Id>XYZABC40S18L424B</Id>
         </CountryCode>
      </Country>
   </TaxCode>
</OrgId>

To learn how to pass a tag identifier to a data masking routine, see XML Masking Template UDCXML1 .

Parent topic: Masking


Provided Templates

Data Express provides the following templates:

For the UNLOAD format
  • UDCXML1 is a prototype for masking XML data
  • UDCXML0 is a prototype for managing data stored in large object columns not needing masking
  • UDCLOB1 is a prototype for masking positional data in a large object column
For the DSNTIAUL format
  • UDCXML6 is a prototype for masking XML data
  • UDCLOB6 is a prototype for masking positional data in a large object column
Note: When using the DSNTIAUL format with LOB columns that are not masked, a masking routine specification is not required.
  • XML Masking Template UDCXML1
  • Dummy Masking Template UDCXML0
  • LOB/BLOB/CLOB Masking Template UDCLOB1
Parent topic: Masking


XML Masking Template UDCXML1

Data Express provides the UDCXML1 data masking routine as a template for processing XML data. You can use the template as-is, or customize and/or replicate it according to your needs.

Note: The information in this topic that applies to UDCXML1 also applies to UDCXML6 except for the output PDS naming convention because PDSs are not present in DSNTIAUL format. In addition, the information in this topic that applies to UDCLOB1 also applies to UDCLOB6.

The sections that follow explain how to customize the STARTING-OPERATIONS section in the UDCXML1 template to manage:

  • Character set
  • Passing XML data
  • XML masking routines
  • The naming convention for output PDSs

Character Set

Source XML can be stored in DB2 using any of several supported character sets such as EBCDIC, ASCII, various dialects of EBCDIC and ASCII, or even custom character sets.

To correctly manage this, the engine called by the data masking routine needs to know the character set in use so it can appropriately process the data. Depending on the specified character set, the data masking engine performs certain tasks such as the transforming of single-byte characters from one set to another, transforming EBCDIC to ASCII before processing, and recoding any transformed characters back to the original character set before writing the output.

Standard character set

By default, the character set specified to Data Express is "*NONE", meaning that no character transforming is required. This assumes that the data being parsed is in EBCDIC and uses the same code page as is specified in your Data Express mainframe installation:

       STARTING-OPERATIONS.
      *
      * CHARACTER SET
      *
      *    MOVE "ASCII"              TO WS-CHARSET.
      *    MOVE "EBCDIC"             TO WS-CHARSET.
           MOVE "*NONE"              TO WS-CHARSET.

In addition to ASCII and EBCDIC, you can customize this value to represent an alternative or a custom character set.

Alternative or customized character set
To specify a character set other than ASCII or EBCDIC, you must change the value passed to the WS-CHARSET variable. To do this, edit the character set conversion logic in the provided KDCXML3W copybook, which is called by the UDCXML1 routine. Follow these steps:
  1. Add 1 to the KDCXML3-NUM-TAB-USATE VALUE clause.
  2. After the existing TRASC02 group field, add a new similarly-coded group field named TRASC03.
    Note: If TRASC03 or any number of additional TRASC0n fields already exist due to previous editing, name the new group field by incrementing n to the next consecutive number; for example, if TRASC03 exists, name the new group field TRASC04.
  3. Code the new group field as follows:
    • An initial elementary field that is the name of the encoding.
    • A set of variables that associate the value of a character in the customized character set with the value of the corresponding character in the EBCDIC character set.

    For example:

                 10 TRASC03.
                15 TAB-TRASC03 PIC X(16) VALUE 'NO-TRASX'.
                15 EL-TRASC03-001 PIC X(2) VALUE X'0000'.
                15 EL-TRASC03-002 PIC X(2) VALUE X'0101'.
                15 EL-TRASC03-003 PIC X(2) VALUE X'0202'.
                .
                .
                .
                15 EL-TRASC03-256 PIC X(2) VALUE X'0000'.
  4. Following the new code group, locate and delete the following rows:
                 10 TAB-VALUE03 PIC X(16).
             10 TAB-EL03 PIC X(2) OCCURS 256 TIMES.
    Note: If your group field name uses an incremented number other than 3, find and delete the two rows that correspond to the number used for your group field.
  5. Recompile UDCXML1.

Passing XML Data to Masking Routines

To pass data to a masking routine, you code several MOVE statements in the STARTING-OPERATIONS section. Before starting:

  • Assign a tag identifier to each data masking object in your XML code. See XML Tag Identifiers for details.
  • Determine the total number of data masking object instances contained in your XML code. A data masking object instance is single occurrence of a data masking object.
  • Associate the number 1 with the first data masking job instance, and continue numbering consecutively through the last instance. These numbers are for your reference.
  • Determine the maximum length of each data masking object.
  • Determine the name of each data masking routine you want to use. See Masking Routines for more information.

Data Express provides the WS-NUMTAG(n) field to hold the total number of data masking object instances to process, and provides the following three fields to identify the data and the masking routine for each instance. Each data masking object instance uses all three fields:

WS-TAG(n)
Where n is the number (from 1 to WS-NUMTAG) that you have associated with the data masking object instance. Into this field, you move the tag identifier for a data masking object instance.
WS-LEN1(n)
Where n is the number (from 1 to WS-NUMTAG) that you have associated with the data masking object instance. Into this field, move the length of the data masking object instance. You can set this to an exact field length for standard-sized data, such as an IBAN, or set it higher to accommodate dynamic field lengths. The limit is 255.
WS-ROUTINE(n)
Where n is the number (from 1 to WS-NUMTAG) that you have associated with the data masking object instance. Into this field, move the name of the masking routine to use.

The following example shows code used to move the data from eight XML data object instances to their appropriate masking routines:

      *---------------------------------------------------------------
      *
       STARTING-OPERATIONS.
          .
          .
          .
      *
      * MASKING ROUTINE
      *
           MOVE  8                   TO WS-NUMTAG.
           MOVE "Nm"                 TO WS-TAG      (1).
           MOVE 255                  TO WS-LEN1     (1).
           MOVE "ADLER"              TO WS-ROUTINE  (1).
           MOVE "AdrLine"            TO WS-TAG      (2).
           MOVE 255                  TO WS-LEN1     (2).
           MOVE "ADLER"              TO WS-ROUTINE  (2).
           MOVE
           "Mndt,Dbtr,Id,PrvtId,Othr,Id"
                                     TO WS-TAG      (3).
           MOVE 16                   TO WS-LEN1     (3).
           MOVE "UDCCDFC"            TO WS-ROUTINE  (3).
           MOVE SPACES               TO WS-TAG      (4).
           STRING "UndrlygAccptncDtls,OrgnlMndt,OrgnlMndt,"
                   DELIMITED BY SIZE
                  "Dbtr,Id,PrvtId,Othr,Id"
                   DELIMITED BY SIZE
                                   INTO WS-TAG      (4).
          MOVE 16                   TO WS-LEN1     (4).
          MOVE "UDCCDFC"            TO WS-ROUTINE  (4).
          MOVE "BirthDt"            TO WS-TAG      (5).
          MOVE 10                   TO WS-LEN1     (5).
          MOVE "UDCMDY0"            TO WS-ROUTINE  (5)
          MOVE "NOME"               TO WS-TAG      (6).
          MOVE 255                  TO WS-LEN1     (6).
          MOVE "UDCNAMC"            TO WS-ROUTINE  (6)
          MOVE "COGNOME"            TO WS-TAG      (7).
          MOVE 255                  TO WS-LEN1     (7).
          MOVE "UDCSURC"            TO WS-ROUTINE  (7).
          MOVE "TAXCODE"            TO WS-TAG      (8).
          MOVE 16                   TO WS-LEN1     (8).
          MOVE "UDCCDFC"            TO WS-ROUTINE  (8).

      STARTING-OPERATIONS-EX.

Output PDS Naming Convention

You can customize the following algorithm to set the output PDS. The name of the variable to use is WS-PDS-OUT:

      *---------------------------------------------------------------
      *
       STARTING-OPERATIONS.
          .
          .
          .
      *
      * OUTPUT PDS NAMING CONVENTION
      *
           MOVE SPACES TO WS-PDS-OUT.
           STRING "XXX.YYY." DELIMITED BY SIZE
                  WS-METODO DELIMITED BY SPACES
                  "." DELIMITED BY SIZE
                  WS-TABELLA DELIMITED BY SPACES
                  ".LIB" DELIMITED BY SIZE
                  INTO WS-PDS-OUT.
Parent topic: Provided Templates


Dummy Masking Template UDCXML0

In addition to the UDCXML1 XML masking template, Data Express also provides the UDCXML0 XML masking template. Use this simple template when character set and data masking object customizations are not required. You can edit UDCXML0 to customize the output PDS naming convention just as you would in UDCXML1. For details, see the Output PDS Naming Convention section in XML Masking Template UDCXML1.

Parent topic: Provided Templates


LOB/BLOB/CLOB Masking Template UDCLOB1

Data Express provides the UDCLOB1 data masking routine as a template for managing specific positional LOB/BLOB/CLOB cases. You can use the template as-is, or customize and/or replicate it according to your needs.

The sections that follow explain how to customize the STARTING-OPERATIONS section in the UDCLOB1 template to manage:

  • The starting and ending position for each masking object
  • The naming convention for output PDSs

Passing LOB/BLOB/CLOB Data to Masking Routines

To pass data to a masking routine, you code several MOVE statements in the STARTING-OPERATIONS section. Before starting:

  • Determine the total number of data masking object instances contained in your LOB/BLOB/CLOB code. A data masking object instance is single occurrence of a data masking object.
  • Associate the number 1 with the first data masking job instance, and continue numbering consecutively through the last instance. These numbers are for your reference.
  • Determine the starting position for each data masking object.
  • Determine the ending position for each data masking object.
  • Determine the name of each data masking routine you want to use. See Masking Routines for more information.

Data Express provides the WS-NUMTAG(n) field to hold the total number of data masking object instances to process, and provides the following three fields to identify the data and the masking routine for each instance. Each data masking object instance uses all three fields:

WS-POS1(n)
Where n is the number (from 1 to WS-NUMTAG) that you have associated with the data masking object instance. Into this field, you move the starting position for a data masking object instance.
WS-END1(n)
Where n is the number (from 1 to WS-NUMTAG) that you have associated with the data masking object instance. Into this field, you move the starting position for a data masking object instance.
WS-ROUTINE(n)
Where n is the number (from 1 to WS-NUMTAG) that you have associated with the data masking object instance. Into this field, move the name of the masking routine to use.

The following example shows code used to move the data from three LOB/BLOB/CLOB data object instances to their appropriate masking routines. This code represents:

  • A name at position 1 with length 20
  • A surname at position 21 with length 20
  • A tax code at position 190 with length 16
      *---------------------------------------------------------------
      *
       STARTING-OPERATIONS.
          .
          .
          .
      *
      * MASKING ROUTINE
      *
          MOVE 3                    TO WS-NUMTAG.
          MOVE 1                    TO WS-POS1     (1).
          MOVE 20                   TO WS-END1     (1).
          MOVE "UDCNAMC"            TO WS-ROUTINE  (1).
          MOVE 21                   TO WS-POS1     (2).
          MOVE 40                   TO WS-END1     (2).
          MOVE "UDCSURC"            TO WS-ROUTINE  (2).
          MOVE 190                  TO WS-POS1     (3).
          MOVE 205                  TO WS-END1     (3).
          MOVE "UDCCDFC"            TO WS-ROUTINE  (3).

      STARTING-OPERATIONS-EX.

Output PDS Naming Convention

You can customize the following algorithm to set the output PDS. The name of the variable to use is WS-PDS-OUT:

      *---------------------------------------------------------------
      *
       STARTING-OPERATIONS.
          .
          .
          .
      *
      * OUTPUT PDS NAMING CONVENTION
      *
           MOVE SPACES TO WS-PDS-OUT.
           STRING "XXX.YYY." DELIMITED BY SIZE
                  WS-METODO DELIMITED BY SPACES
                  "." DELIMITED BY SIZE
                  WS-TABELLA DELIMITED BY SPACES
                  ".LIB" DELIMITED BY SIZE
                  INTO WS-PDS-OUT.
Parent topic: Provided Templates


Data Masking Guide

Outlines the concepts and procedures used by the Data Masking module. Data Masking makes sensitive data anonymous in application test environments while maintaining the same characteristics as the original data.

The Data Masking module comprises a mainframe engine and a client configuration tool. The mainframe engine reads and catalogs the original information, and stores it in the Knowledge Base.

Data Masking enables seamless access to supported rDBMSes, by providing the additional connectivity, infrastructure, and interfaces for this environment.

Data Masking processing comprises the following elements:

  • Setup processing that automatically identifies file layouts.
  • The association of routines with data elements to mask data and create the modified data stores.
  • Who Should Read this Guide
  • Data Masking Considerations
  • Getting Started
    Provides an overview of the Data Masking module. You should read it before using the module.
  • Data Masking Main Window
  • Data Changer Data Store Properties
  • Data Changer Data Element Selection
  • Data Changer Data Element Properties
  • Data Changer - Work with Classes
  • Data Changer Properties
  • Set Application ID
  • Define Unload Output Data Store
  • Options
  • Work with Data Changer Jobs
  • Distributed Exporter
  • Exit Routines
  • Appendix A: COBOL Masking Routines Technical Description
  • Appendix B: C Masking Routines for ODBC - Technical Description
  • Appendix C: C Masking Routines for OCI - Technical Description
Parent topic: Data Express


Who Should Read this Guide

This guide is for Micro Focus users who are interested in managing the privacy of data contained in databases of applications running on z/OS or distributed platforms that are either developed internally or purchased by external suppliers.

This guide explains the Data Masking configuration procedure, and describes the steps required to migrate the data stores in your application.

Before using this product, we recommend you analyze all the files to be processed and associate them with their copybooks using the Data Express Data module. We also recommend that you carefully read the Process Guide for z/OS and Front End Guide.

Parent topic: Data Masking Guide


Data Masking Considerations

Bear in mind the following points about the Data Masking module for Data Express:

  • This product is designed to read data from data stores containing the information to be analyzed, and to rewrite the content of these data stores to output data stores specified by the user.
  • To use this product with z/OS, you must have the correct licence. See the Installation Guide for more details.
  • You do not need to analyze the sources of the programs in the application in order to use Data Masking.
  • Data Masking does not perform any operation capable of altering the analyzed data.

The tables defined during the installation phase that are updated by Data Masking are described in Appendix A. Data Masking Tables.

Parent topic: Data Masking Guide


Getting Started

Provides an overview of the Data Masking module. You should read it before using the module.
  • General Characteristics of Use
  • Starting Data Masking
  • Menus and Toolbars
Parent topic: Data Masking Guide


General Characteristics of Use

Data Express for z/OS can be used in either Standard or Client/Server mode, whereas Data Express for Distributed Systems can only be used in Standard mode:

  • Standard – Supports direct access to the data resident on RDBMS from your PC.
  • Client/Server – Supports access to the data resident on RDBMS through the kbde-Server: a product based on a three-tier architecture where the second tier handles data transfer optimization. Client/Server mode lets you access data over the Internet with the lowest amount of administration necessary and the best utilization of network capacity.
Note: Client/Server mode can use the same functions as Standard mode without you having to install additional software on your PC, such as the DB2 client or the Borland Database Engine (BDE).

For Data Express configuration information and instructions, see:

  • Appendix A. Standard Mode in the Front End Guide.
  • Appendix B. Client/Server Mode in the Front End Guide and in Appendix C. Client/Server Mode of this guide.

The following guidelines apply to windows and screen items within the Data Masking module:

  • The main window: Data Masking - [Active Files] cannot be closed except when exiting the module.
  • Multiple MDI windows may be opened simultaneously.
  • Right-click an item or area to view a list of applicable commands.
Parent topic: Getting Started


Starting Data Masking

To start the Data Masking module:

  1. Click Start > All Programs > Micro Focus Data Express 4.0 > Data Masking.
    Note: If you are using Data Express on Windows Vista and User Access Control is enabled, you must run Data Masking as Administrator.
  2. Select the required database with which to launch the connection. By default, the highlighted item in the list of available databases is the last database you connected to using the Data Masking module.
    Note: To connect to the database for first time, you need your user ID and password.

Select the required schema. The Main Window appears as shown:

Figure 1. Data Masking - Main Window

Tip: Once you are connected to a database, you can open and close a database from the Main Window by clicking File > Open Database > File > Close Database.
Parent topic: Getting Started


Menus and Toolbars

This section describes the menus and common toolbars displayed in the Main Window.

  • File Menu
  • Tools Menu
  • View Menu
  • Windows Menu
  • Help Menu
  • Toolbar
  • Context Menus
Parent topic: Getting Started


File Menu

The File menu contains the following commands:

  • Open Database – Opens a new database connection
  • Close Database – Closes database connection
  • Exit – Terminates the Data Masking module
Parent topic: Menus and Toolbars
Related information
Tools Menu
View Menu
Windows Menu
Help Menu
Toolbar
Context Menus


Tools Menu

The Tools menu contains the following commands:

  • Enable All Data Stores – Specifies to enable all disabled data stores for the currently selected machine ID and company combination that have data elements assigned to the data masking routine.
  • Options – Opens the Option window for setting the help language, security options, and database user ID and password.
  • Connection Options – Opens the Connection Option window for changing the type of connection.
Parent topic: Menus and Toolbars
Related information
File Menu
View Menu
Windows Menu
Help Menu
Toolbar
Context Menus


View Menu

The View menu contains the following commands:

  • Hide ToolBar – Displays or hides the tool bar
  • Hide Status Bar – Displays or hides the status bar
Parent topic: Menus and Toolbars
Related information
File Menu
Tools Menu
Windows Menu
Help Menu
Toolbar
Context Menus


Windows Menu

The set of open windows can be organized according to the standard Windows arrangements. The Windows menu contains the following commands:

  • Tile Horizontally – Arranges all open windows from top to bottom.
  • Tile Vertically – Arranges all open windows from left to right.
  • Cascade – Arranges all open windows so that they will not overlap.
  • Minimize All – Minimizes all open windows with the exception of the main window.
  • Arrange – Arranges all minimized windows.
Parent topic: Menus and Toolbars
Related information
File Menu
Tools Menu
View Menu
Help Menu
Toolbar
Context Menus


Help Menu

The Help menu contains the following commands:

  • Help Topics, Index – Opens online help information for the product in the language selected in the Options window.
  • Micro Focus SupportLine – Contains commands for connecting to Micro Focus Product Support for information or technical assistance.
  • Micro Focus - Data Express Home Page – Connects to the Micro Focus Data Express web site.
  • About – Displays production information for the Data Masking module.
Parent topic: Menus and Toolbars
Related information
File Menu
Tools Menu
View Menu
Windows Menu
Toolbar
Context Menus


Toolbar

The icons listed in default toolbars are described as follows:

  • Open database connection - Open database connection
  • Close database connection - Close database connection
  • Refresh information - Refresh information
  • Work with classes - Work with classes
  • Distributed Exporter - Distributed Exporter
Parent topic: Menus and Toolbars
Related information
File Menu
Tools Menu
View Menu
Windows Menu
Help Menu
Context Menus


Context Menus

This section describes the commands included in context menus.

  • Column Sorting
  • Grid Functions
Parent topic: Menus and Toolbars
Related information
File Menu
Tools Menu
View Menu
Windows Menu
Help Menu
Toolbar


Column Sorting

A context menu to display a list of sort options can be displayed by right-clicking a column header.

Note:

All changes to the grid layout are saved in the Windows system registry. For example, groups, column width attributes, column order attributes, and column font and color information. However, filtering and sorting preferences are not saved.

The column sorting context menu contains the following commands:

  • Sort Descending – Sorts the list contents in descending order. The same result can be obtained by simply clicking the column heading.
  • Sort Ascending – Sorts the list contents in ascending order. The same result can be obtained by simply clicking the column heading.
  • Group by This Column – Groups the list according to the contents of the selected column if the Group Panel has been enabled.
  • Remove This Column – Deletes the selected column.
  • Column Selector – Displays a Customize window that contains a list of columns that have been removed previously. A column can be restored by performing a drag-and-drop operation to the desired position in the list. A column can be removed from a list by performing a drag-and-drop operation from the list to the Customize window.
  • Alignment – Aligns the content of the columns to the center, right, or left as specified by the user.
  • Best Fit – Resizes the selected column to allow the content of each record to be displayed. The same result can be obtained by double-clicking the column’s external margin.
  • Pick Color – Allows the color of the selected column to be changed.
  • Pick Font – Allows the font, font style, size, color, and script to be changed.
  • Best Fit (All Columns) – Resizes all the columns from the list to allow the content of each record to be displayed.
    Note: Columns can also be sorted by clicking or performing a drag-and-drop operation on the column heading.
  • Information displayed can be reduced by clicking a column heading, and then typing the first letter of the items you are interested in.
Parent topic: Context Menus


Grid Functions

A context menu to display a list of functions for a grid can be displayed by right-clicking any place on the grid.

Note: All changes to the grid layout are saved in the Windows registry. For example, groups , column width attributes, column order attributes , and column font and color information . However, filtering and sorting preferences are not saved.

The grid functions context menu contains the following commands:

  • Column Selector - Displays a Customize window that contains a list of columns that have been removed previously. A column can be restored by performing a drag-and-drop operation to the desired position in the list. A column can be removed from a list by performing a drag-and-drop operation from the list to the Customize window.
  • Best Fit (All Columns) - Resizes all the columns from the list to allow the content of each record to be displayed.
  • Export to File… - Exports the information to a file using one of the following compatible formats:
    • comma separated value (.csv)
    • Hyper Text Markup Language (.htm)
    • text (.txt)
    • Excel (.xls)
    • Extensible Markup Language (.xml)
  • Show Summary Footer - Displays the summary footer beneath the grid. By selecting a row and right-clicking the summary footer, a context menu displaying arithmetic operations you can perform on the selected row appears.
    Note: Arithmetic operations are only possible on numeric contents.

    Possible arithmetic operations include:

    • Summary - Displays the sum of all quantities in the column . T his option is active only for columns with integer or percentage values ; if the value of the cells in a specific column are represented as a percentage, their sum does not always equal 100% .
    • Minimum - Displays the minimum value of all values in the column. The option is active only for columns with numerical values.
    • Maximum - Displays the maximum value of all values in the column. The option is active only for columns with numerical values.
    • Count - Displays the number of elements in a column.
    • Average - Displays the arithmetic mean of all values in the column. The option is active only for columns with numerical values.
    • None - Specifies using no arithmetic operation on a column.
  • Show Group Panel - Displays the column name by which data is grouped above the grid. By dragging the column headers into this window, a group hierarchy can be defined.
    Note: If data is being grouped by column, a footer panel is displayed beneath the grid for each group.
  • Filter Active - If enabled, lets you choose data to be displayed on the grid; a downward arrow icon will appear in the column headers.

    Click an arrow to choose the data to be displayed on the grid. This function lets you generate a personal filter.

Note: The time taken to display the window increases with the number of records to be loaded.
Parent topic: Context Menus


Data Masking Main Window

  • Introduction
  • Window Components
  • Column Headings in the Grid Area
  • Available Grid Operations
Parent topic: Data Masking Guide


Introduction

After starting Data Masking, the window: Main Window appears.

From this window, you can enable or disable data stores, and view a report of how the classification or the partitioning of your data stores is being performed for the specified Machine ID and Company names.

Parent topic: Data Masking Main Window


Window Components

This section describes each component of the Main Window area.

The Main Window comprises four main areas: Machine ID./Company grid, All Machine ID./Companies, Multiple Class Selection, and Object Distribution as shown here:

Data Masking Main Window
  • Machine ID./Company Grid
  • All Machine ID./Companies
  • Multiple Class Selection
  • Object Distribution
Parent topic: Data Masking Main Window


Machine ID./Company Grid

The area containing the grid and the tabs Enabled and Disabled is referred to as the Machine ID./Company area or the Grid area. The grid lists data stores related to the selected class and to the selected machine ID and company. The names of the forms, their meaning and the included information are outlined below.

The Enabled tab shows a list of all active data stores (enabling a data stores means that it is visible in the export functionality in order to be prepared for masking), and all data stores, which have one or more specific classes related to their data elements.

Select and right-click a data store from the list to view a list of available commands as listed below:

  • Open Selected Data Store – Displays a window with information about the data elements of the data store.
  • Data Store Properties – Displays a window with all the information about the data store.
  • Enable or Disable – Enables or disables the masking function.
  • Data Changer properties – Displays a window with information about the Data Changer function for the selected data store.
  • Set Application ID – Open a window to select the application identifier.
Note: The Data Changer (also referred to as the Data Masker) is the function within Data Masking that performs the masking.
  • The Work with Data Changer Files function is available in Data Builder.
  • For more information on working with the Data Changer function, see the Front End Guide.
Parent topic: Window Components


All Machine ID./Companies

The Machine ID./Companies area contains drop-down lists from which you can choose the machine ID and the company ID for the specified assigned class. The hierarchical structure shows the grouping associated with each root-level node. Click All Files or select the assigned class you are interested in, and use the Enabled and Disabled tabs to view a list of active or inactive data stores respectively in the Grid area (which is part of the Machine ID./Company area).

Parent topic: Window Components


Multiple Class Selection

The Multiple Class Selection area enables you to drag and drop two or more classes into it for class selection. Click the appropriate (x) icon to remove a class.

Tip: The same function can also be executed from a context menu, which is accessible by right-clicking this area.
Parent topic: Window Components


Object Distribution

The Object Distribution area displays a pie chart and bar chart representation of the data store types for the classes selected in the Machine ID./Companies or Multiple Class Section area. The pie chart shows the percentage distribution of data store types, while the bar chart shows the numerical distribution of data store types. For both charts, both enabled and disabled data stores are included. Click the minimize button (_) to minimize the Object Distribution area.

Parent topic: Window Components


Column Headings in the Grid Area

The following list describes the column headings in the Grid area for data stores that have been loaded in the Data Express Knowledge Base:

  • Machine ID – Identifier of the machine.
  • Company – Name of the company.
  • Data Store Name – Name of the data store.
  • Data Store type – Type of data store. For example, DB2, VSAM, DL/I, GDG, SEQ, ODBC, or ORA.
  • Record Format – Name of the record format.
  • Application ID – Identifier (if it exists) of the application area the data store belongs to.
  • Number of Records – Number of records contained in the data store.
  • Record Length – Length (in bytes) of the record.
  • Unload Input Data Store Type – Type of the Unload Input data store.
  • Unload Input Data Store Name – Name of the Unload Input data store.
  • Unload Input Data Store Version – Version of the Unload Input data store.
  • Format Selector – Exit routine for data store format recognition.
  • Select Program – Selection program of the individual record.
  • Unload Output Data Store Type – Type of the Unload Output data store.
  • Unload Output Data Store Name – Name of the Unload Output data store.
  • Unload Output Data Store Version – Version of the Unload Output data store.
  • Assigned Copy PDS – Name of the PDS, which contains the copybook associated with the data store.
  • Assigned Copy Name – Name of the copybook associated with the data store.
  • Database – Name of the database.
Parent topic: Data Masking Main Window


Available Grid Operations

Right-click a data store name in the grid to view a list of available commands as described below:

  • Open the Selected Data Store – Displays a window that lists all data elements as well as the classes assigned to them.
  • Data Store Properties – Displays a window listing data store information.
  • Data Changer Properties – Displays a window listing information for the associated Data Changer routine.
  • Disable or Enable – Enables a data store (if it is actively disabled) or disables a data store (if it is actively enabled).
  • Set File Category – Displays a simple form to specify data store category and a brief description of the category for the selected files in the Machine Id./Companies or Multiple Class Section area.
  • Set Application ID – Displays a simple form to specify the application ID and a brief description of the ID.
Parent topic: Data Masking Main Window


Data Changer Data Store Properties

The Properties - File window is a tabbed view that displays all attributes for a selected data store.

  • Introduction
  • Window Contents
Parent topic: Data Masking Guide


Introduction

To access the Properties - File window, right-click the file in the Grid area of the Data Masking main window, and then click Data Store Properties.

For more information about the context menu, see Machine Id./Company Grid in the Data Masking Main Window section.

Parent topic: Data Changer Data Store Properties
Related concepts
Machine ID./Company Grid


Window Contents

This section describes the fields listed on each tab of the Properties – File window as shown below:

Figure 1. Properties – File Window Listing File Attributes
Properties - File: CUSTOMER
Note: The Masking Input and Masking Output tabs display only when a Data Masking routine is associated with the data store. For more information about Changer Program properties, see Window Contents in the Data Changer Properties section.
  • General Attributes
  • System
  • Object Attributes
  • Dates
  • Assigned Copybook
Parent topic: Data Changer Data Store Properties
Related concepts
Window Contents


General Attributes

The fields listed on the General Attributes tab are:

  • Data Store Description – Description of the data store.
  • Appl. ID – Identifier (if it exists) of the application area that the data store belongs to.
  • Process ID – Process identifier.
  • Format selector – Exit routine for file format recognition.
  • Record length – Length (in bytes) of the record.
  • Data Store name – Name of the data store.
  • Data Store type – Type of data store. For example, DB2, VSAM, DL/I, GDG, SEQ, ODBC, or ORA.
  • Data Store version – Version of the data store.
  • Sampling frequency – Frequency (for example, daily or monthly) of sampling of the data elements for which the distribution of values is desired.
  • v – Indicator of whether the data store contains demographic data.
Parent topic: Window Contents


System

The fields listed on the System tab are:

  • Machine ID – Identifier of the machine
  • Company name – Name of the company
Parent topic: Window Contents


Object Attributes

The fields listed on the Object Attributes tab are:

  • Data Store type – Type of data store. For example, DB2, VSAM, DL/I, GDG, SEQ, ODBC, or ORA.
  • Data Store name – Name of the data store.
  • Data Store description – Description of the data store.
  • Application ID – Identifier (if it exists) of the application area that the data store belongs to.
  • Record length – Length (in bytes) of the record.
  • Record format – Name of the record format.
  • Format selector – Exit routine for file format recognition.
  • Number of records – Number of records contained in the data store.
  • Analyzed records – Number of analyzed records.
  • Database name – Name of the database.
  • Tablespace name – Name of the tablespace.
  • Object ID – Object identifier.
Parent topic: Window Contents


Dates

The fields listed on the Dates tab are:

  • Date of Analysis – Date of the analysis carried out on the data store.
  • Date of Last Calculation – Date of the last recalculation carried out on the data store.
  • Date of Last Sample – Date of the last calculation of the distribution of values of the data elements involved in the process.
Parent topic: Window Contents


Assigned Copybook

The fields on the Assigned Copybook tab are:

  • Copybook PDS – Name of the area (level 01) within the copybook associated with the data store.
  • Copybook name – Name of the copybook assigned to the data store.
  • I/O Area Name – Name of the area present on the copybook that describes the layout of the file.
  • I/O area progressive – Progressive number of the area within the copybook associated with the file.
  • Information provider – Supplier of the association information; PGM is information obtained from batch processing, USR is information supplied by the user.
Note: The Assigned Copybook tab is not visible for DB2 tables.
Parent topic: Window Contents


Data Changer Data Element Selection

The Data Store Information tab in the Main Window is similar to the Work with Data Elements area in Data Builder. Both areas can be used to associate modification routines to a data element in order to enable the routine associated with the data element, and to view the information about the data element. However, the Data Store Information tab filters information for the selected data store by the attributes Machine ID, Company ID, Data Store Type, and Data Store Name.

  • Introduction
  • Window Contents
Parent topic: Data Masking Guide


Introduction

The Data Store Information tab can be used to associate modification routines to a data element in order to enable the routine associated with the data element, and to view the information about the data element. This tab supplies all available information about the selected data store and all the classes associated with it.

To access the Data Store Information tab, right-click the data store in the Grid area of the Data Masking main window, and click Open Selected Data Store.

For more information about the context menu, see Machine Id./Company Grid in the Data Masking Main Window section.

Parent topic: Data Changer Data Element Selection
Related concepts
Machine ID./Company Grid
Introduction


Window Contents

This section describes the fields and items listed in the Data Store Information area as shown below:

Figure 1. Data Masking – Data Store Information Tab

  • Data Store Information
  • Icon Bar
  • List of Data Elements
Parent topic: Data Changer Data Element Selection


Data Store Information

The fields listed in the Data Store Information tab are:

  • Machine ID – Identifier of the machine
  • Company – Name of the company.
  • Data Store type – Type of data store. For example, DB2, VSAM, DL/I, GDG, SEQ, ODBC, or ORA.
  • Record format – Name of the record format.
  • Data Store Name – Name of data store.
  • Number of Records – Number of records contained in the data store.
  • Register Table – Set/Reset table as Register.
Parent topic: Window Contents
Related information
Icon Bar
List of Data Elements


Icon Bar

The buttons listed in the button bar are:

Icon Bar
  • All Data Elements – Shows all data elements with Data Changer enabled and disabled
  • Enabled Data Elements – Shows the data elements with Data Changer enabled
  • Disabled Data Elements – Shows the data elements with Data Changer disabled
Parent topic: Window Contents
Related information
Data Store Information
List of Data Elements


List of Data Elements

The column names listed in the List of Data Elements tab are:

  • Name – The name of the data element within the data store.
  • Data Element Offset – The offset of the data element.
  • Type – The data type of the data element. For example, Numeric or Alphanumeric.
  • Nature – The nature of the data element. For example, Alphabetical.
  • Integer – The number of integers within a numeric data element.
  • Length – The length of the data element in bytes.
  • Decimal – The number of digits included to the right of the decimal point.
  • Enabled – An indicator of whether the data element modification routines have been enabled (Yes, No).
  • Changer Program – The name of the modification routine associated with the data element.
  • Assigned By – The changer program assignment (By Class, By Field).
  • Assigned Class – Description of the associated class.
Note: A list of available values for the Type column is provided in the following table:
Symbol | Type | Database(s)
1 | Bit | MVS file type, ODBC-enabled*
A | Alphanumeric | MVS file type
B | Binary | MVS file type
C | Combined | MVS file type
D | RowID | Oracle
E | Edited Field | MVS file type
F | Float | MVS file type, Oracle, ODBC-enabled*
G | Long | Oracle
H | Char | Oracle, ODBC-enabled*
I | Integer | ODBC-enabled*
K | CLOB | Oracle
L | Date | MVS file type
M | Money | ODBC-enabled*
N | Numeric | MVS file type, ODBC-enabled*
O | BLOB | Oracle
P | Packed | MVS file type
R | Raw | Oracle
S | BigInt | ODBC-enabled*
T | Time | MVS file type
U | Number | Oracle
V | Varchar | Oracle, ODBC-enabled*
W | Varchar2 | Oracle
X | Text | ODBC-enabled*
Z | Timestamp | MVS file type, Oracle, ODBC-enabled*
b | VarBinary | ODBC-enabled*
d | Decimal | ODBC-enabled*
e | XML | ODBC-enabled*
f | BFile | Oracle
g | Image | ODBC-enabled*
i | TinyInt | ODBC-enabled*
l | DateTime | ODBC-enabled*
m | SmallMoney | ODBC-enabled*
r | Real | ODBC-enabled*
s | SmallInt | ODBC-enabled*
t | SmallDateTime | ODBC-enabled*
u | UniqueIdentifier | ODBC-enabled*
w | Long Raw | Oracle
x | Text | ODBC-enabled*
*For the ODBC-enabled database, the contents of the Type column reflect the data name reported by the ODBC driver.
Parent topic: Window Contents
Related information
Data Store Information
Icon Bar
* For the ODBC-enabled database, the contents of the Type column reflect the data name reported by the ODBC driver.


Data Changer Data Element Properties

This section describes the general and additional Data Changer properties for a selected data element.

  • Introduction
  • General Data Element Properties
  • Additional Data Element Properties
    Additional properties are displayed in the Data Changer Data Element Properties window.
Parent topic: Data Masking Guide


Introduction

Menu commands that enable you to view these properties can be viewed by right-clicking the appropriate class in the grid from the Data Store Information tab in the Main Window.

For more information about the Data Store Information tab in the Main Window, see the Introduction in the Data Changer Data Element Selection section.

Parent topic: Data Changer Data Element Properties
Related concepts
Introduction
Related information
General Data Element Properties
Additional Data Element Properties


General Data Element Properties

General data element properties are displayed in the Data Element Properties window. To display the Data Element Properties window, click Data Element properties from the context menu. An example of the Data Element Properties window is shown below:

Figure 1. Data Element Properties Window

The fields and items listed in the Data Element Properties window are:

  • Name – The name of the data element within the data store.
  • Description – A description of the data store.
  • Type – The type of field. For example, NUMERIC or ALPHANUMERIC.
  • Nature – The nature of the data element. For example, ALPHABETICAL.
  • Displacement – The offset of the data element offset.
  • Length – The length of the data element in bytes.
  • Integer – The number of integers within a numeric data element.
  • Decimal – The number of digits included to the right of the decimal point.
  • Key Field – An indicator of whether the data element is considered a key data element (Yes, No).
  • Register Table Field – An indicator of whether the data element is unique in the data store (Yes, No).
  • Changer Program – The name of the modification routine associated with the data element.
  • Activated changer – An indicator of whether or not the Changer Program has been activated.
  • Field Sign – An indicator of whether the data element is considered signed or not signed.
  • Variable Length – An indicator of whether the length of the data element is variable or not.
  • Justified – An indicator of whether the data element is justified or not.
  • Nullable Field – An indicator of whether the data element is NULL or not.
Parent topic: Data Changer Data Element Properties
Related information
Introduction
Additional Data Element Properties


Additional Data Element Properties

Additional properties are displayed in the Data Changer Data Element Properties window.

To display the Data Changer Data Element Properties window, click Data Changer Data Element Properties from the context menu. An example of the Data Changer Data Element Properties window is shown below:

Figure 1. Data Changer Data Element Properties Window

Data Changer Data Element Properties Window

Note: If data element properties have not been set previously, the data element properties can be edited; otherwise, the properties can only be viewed.

The fields and buttons listed in the Data Changer Data Element Properties window are:

  • Changer Program - The program that will be used when data will be published.
  • Activated Changer - Indicates whether the Data Changer is active or not (Yes, No).
  • Change - Enables control and goes into edit mode.
  • OK – Saves any changes that have not been applied and closes the window.
  • Cancel – Cancels any changes applied to the selected class. Any actions confirmed with the Apply button cannot be restored with Cancel but must be entered and confirmed again.
  • Apply - Confirms the changes.
Parent topic: Data Changer Data Element Properties
Related information
Introduction
General Data Element Properties


Data Changer - Work with Classes

The Work with Classes area enables you to categorize data elements with homogeneous characteristics and to write generalized routines to check data.

  • Introduction
  • Window Contents
  • Available Operations
Parent topic: Data Masking Guide


Introduction

A class identifies a user-defined group of data elements with homogeneous characteristics. Typical classes of data can be quantities, prices, client code, totals, or currency flag. Each data element can be cataloged as a member of a class.

Classes are grouped into sets called Super Classes. The set of Super Classes is as follows:

  • Numbers (N)
  • Codes (C)
  • Date (D)
  • Complemented Date (9)
  • Descriptions (X)
  • Privacy (P)
  • Other (O)
  • None
Note: The Super Class Other is used for new user classes that might not fall into any of the previous Super Classes. Conversely, the Super Class None is used for classes that do not belong to a SuperClass.

To access the Work with Classes area, click [image] (Work with Classes) from the Data Masking main window.

Parent topic: Data Changer - Work with Classes
Related information
Window Contents
Available Operations


Window Contents

This section describes the fields and items listed in the Work with Classes area as shown below:

Figure 1. Work with Classes Window

Work with Classes Window

The columns and fields listed in the Work with Classes window are:

  • Class ID – Identifier of the class.
  • Super Class – SuperClass that has been associated with the class.
  • Name – Name of the class.
  • Charcode – Character identifier of the class.
  • Description – Brief description of the class. The information in this column matches the information in the Full Description field.
  • Highlight – Primary class flag.
  • Edit Mask – Class editing mask
  • Full Description – Brief description of the class. The information in this field matches the information in the Description column.
  • Data Masking Routine – Program that will be used when the class is assigned as a changer for a selected data element.
Parent topic: Data Changer - Work with Classes
Related information
Introduction
Available Operations


Available Operations

The buttons at the bottom of the Work with Classes window are:

  • Refresh – Refreshes the information listed in the grid.
  • Delete – Deletes the Changer Program for the selected class.
  • OK – Saves any changes that have not been applied and closes the window.
  • Cancel – Cancels any changes applied to the selected class. Any actions confirmed with the Apply button cannot be restored with Cancel but must be entered and confirmed again.
  • Apply – Confirms the changes made to the Changer Program of the selected class.
Parent topic: Data Changer - Work with Classes
Related information
Introduction
Window Contents


Data Changer Properties

The Data Changer properties window lets you view Data Changer information for an enabled data store, or lets you define information for a newly enabled data store that has no Data Changer information.

  • Introduction
  • Window Contents
  • Available operations
Parent topic: Data Masking Guide


Introduction

To access the Data Changer properties window, right-click the data store in the Grid area of the Data Masking main window, and click Data Changer properties.

For more information about the context menu, see Machine ID./Company Grid in the Data Masking Main Window section.

Parent topic: Data Changer Properties
Related concepts
Machine ID./Company Grid
Related information
Window Contents
Available operations


Window Contents

This section describes the fields listed in the Data Changer properties window as shown below:

Figure 1. Data Changer properties Window

The fields listed in the Data Changer properties window are:

  • General Properties
    • Format selector – Name of the exit routine for the format recognition.
    • Write program – Write program of the Output file.
    • Process ID – Identifier of the process.
    • Select program – Selection program of the individual record.
  • Input Data Store Properties
    • Unload data store type – Type of the Unload Input data store.
    • Unload data store name – Name of the Unload Input data store. This is for the z/OS environment only. It can be different from the original name loaded into the Knowledge Base. In order to do this, the process ID associated with the elaboration should be created with characteristics for an Unload File Access (For example: GENUN).
    • Unload data store version – Version of the Unload Input data store.
  • Unload Data Store Properties
    • Unload data store type – Type of the Unload Output data store.
    • Unload data store name – Name of the Unload Output data store.
    • Unload data store version – Version of the Unload Output data store.
Parent topic: Data Changer Properties
Related concepts
Window Contents
Related information
Introduction
Available operations


Available operations

The buttons in the Output Data Store Properties section of the Data Changer properties window are:

  • Change – Enables control and goes into edit mode.
  • OK – Saves any changes that have not been applied and closes the window.
  • Cancel – Cancels any changes. Any actions confirmed with the Apply button cannot be restored with Cancel but must be entered and confirmed again.
  • Apply – Confirms the changes.
Parent topic: Data Changer Properties
Related information
Introduction
Window Contents


Set Application ID

The Set Application ID window is used to set an application ID to one or more selected data stores.

  • Introduction
  • Window Contents
  • Available Operations
Parent topic: Data Masking Guide


Introduction

To access the Set Application ID window, right-click the data store in the Grid area of the Data Masking main window, and click Set Application ID.

For more information about the context menu, see Machine Id./Company Grid in the Data Masking Main Window section.

Parent topic: Set Application ID
Related concepts
Machine ID./Company Grid
Related information
Window Contents
Available Operations


Window Contents

This section describes the fields listed in the Set Application ID window as shown below:

Figure 1. Set Application ID

Set Application ID

The fields listed in the Set Application ID window are:

  • Application ID – Available application identifier. If *NONE is selected, all selected data stores will have no associated application ID.
  • Description – Brief description of the selected application ID.
Parent topic: Set Application ID
Related information
Introduction
Available Operations


Available Operations

The buttons at the bottom of the Set Application ID window are:

  • OK – Sets the selected application to all selected data stores, and closes the window.
  • Cancel – Cancels any changes. Any actions confirmed with the Apply button cannot be restored with Cancel but must be entered and confirmed again.
Parent topic: Set Application ID
Related information
Introduction
Window Contents


Define Unload Output Data Store

The Define Unload Output Data Store Name window is used to enable one or more selected data stores with a specified output data store name.

  • Introduction
  • Window Contents
  • Available Operations
Parent topic: Data Masking Guide


Introduction

To access the Define Unload Output Data Store window:

  1. From the Data Masking main window, click the Disabled tab.
  2. Select one or more data stores you want to enable.
  3. Right-click to display the context menu, and then click Enable.
Parent topic: Define Unload Output Data Store
Related information
Window Contents
Available Operations


Window Contents

This section describes the fields listed in the Define Unload Output Data Store Name window as shown below:

Figure 1. Define Unload Output Data Store Name Window

Define Unload Output Data Store Name Window

The fields listed in the Define Unload Output Data Store Name window are:

  • Add Prefix –Activates the Prefix box so that a prefix string can be entered, and then added to the beginning of every data store name that does not have associated Data Changer information.
  • Add Suffix – Activates the Suffix box so that a suffix string can be entered, and then added to the end of every data store name that does not have associated Data Changer information.
  • String Replacement – Activates the String to Search and Replace with boxes so that all data store names to do not have associated Data Changer information can be changed to the string listed in the Replace with box.
Parent topic: Define Unload Output Data Store
Related information
Introduction
Available Operations


Available Operations

The buttons at the bottom of the Define Unload Output Data Store Name window are:

  • OK – Enables the selected data stores using the selected criteria, and closes the window.
  • Cancel – Cancels any changes. Any actions confirmed with the Apply button cannot be restored with Cancel but must be entered and confirmed again.
Parent topic: Define Unload Output Data Store
Related information
Introduction
Window Contents


Options

Data Masking lets you customize the product in order to meet the requirements of different users.

  • Introduction
  • General Options
    General options can be edited in the Options window.
  • Connection Options
Parent topic: Data Masking Guide


Introduction

This section describes the general and mandatory options in the Data Masking module.

Parent topic: Options
Related information
General Options
Connection Options


General Options

General options can be edited in the Options window.

To display the Options window, click Tools > Options. The Options window is shown below:

Figure 1. Options Window

The fields listed in the Output window are:

  • Help language – Language used for online help. The Help language list contains the languages available on your PC.
  • Save user and password – Indicator of whether or not you want to save your user ID and password information for the current database connection.
Parent topic: Options
Related information
Introduction
Connection Options


Connection Options

Connection options to the kbde-Server can be edited in the Choose your connection type and settings window. Before opening the project with the Data Masking module, you must define the connection information.

To display the Choose your connection type and settings window, click Tools > Connection Options. An example of the Choose your connection type and settings window is shown below:

Figure 1. Choose your connection type and settings Window

Choose your connection type and settings Window

To define connection settings:

  1. Click the tab that corresponds to your connection type, and complete the fields:
    • Local connection – Client connection to database located on the local machine.
      Note: No additional settings are required for local connections.
    • COM/DCOM connection – Client connection to the Data Express Application Server as specified on the client machine (MIDAS.DLL must be present on the client machine.BDE 5.x and DB2 Client 5.2 or above must be present on the Data Express Application Server machine).
      • HostName – The IP address of the NT server.
      Note: If the NT server and local computer are the same, you can leave this field blank. If the NT server and local computer are not the same, they must belong to the same local network.
    • Socket connection – Client connection using TCP/IP protocol to the Borland Socket Server.
      • Host/IP address – IP address of the Borland socket server.
      • Port – Port number assigned to the Borland socket server.
      Note: For this connection type, the client and server machines do not need to be part of the same local network.
    • Web connection – Client connection using HTTP protocol to the ISAPI-compatible web server.
      • URL – URL for the .dll file installed on the server machine.
      Note: For this connection type, the client and server machines do not need to be part of the same local network.
  2. If desired, check Allow connection pooling to enable the specified number of MS as the minimum time interval for the database call to maintain an active connection when the Data Masking module is idle.
Parent topic: Options
Related information
Introduction
General Options


Work with Data Changer Jobs

The Work with Jobs area contains information about the function related to the execution of the Data Store Data Changer job (which is a batch job that executes the masking routines), and is part of the Data Builder module. These jobs are submitted using the Work with Jobs area of Data Builder.

For more information about the Work with Jobs area, see Work with Jobs in the Front End Guide.

  • Introduction
  • Data Store Data Changer Job Creation
  • Create Data Changer Job Creation
  • Available Operations
Parent topic: Data Masking Guide


Introduction

To access the Work with Jobs area:

  1. Start Data Builder.
  2. Click [image] (Work with Jobs) in the toolbar of the Data Builder main window.
    The Work with Jobs area is displayed as shown below:
    Figure 1. Work with Jobs Area

    Work with Jobs Area

Parent topic: Work with Data Changer Jobs
Related information
Data Store Data Changer Job Creation
Create Data Changer Job Creation
Available Operations


Data Store Data Changer Job Creation

The Data Store Data Changer job executes a process to create a JCL that will execute data changing associated with a business protection (masking) procedure. The migration function modifies and transfers the content of an input sequential file into an output sequential file.

To create a Data Store Data Changer job:

  1. From the Work with Jobs window, press the New button. The Work with Jobs window lets you insert parameters for the new job.
  2. In the List of Jobs section, select Data Store Data Changer.
  3. In the General section, select the appropriate Machine ID and Company name.
  4. In the Data Store Qualification section, specify values for the following fields:
    • Application ID – The application ID, if applicable.
    • DB2 Owner, DL/I DBD – The owner of DB2 or DL/I DBD data store where data will be masked.
    • Data Store type – Type of data store. For example, DB2, VSAM, DL/I, GDG, SEQ, ODBC, or ORA.
    • Data store name – The name of the data store where data will be masked.
    • Data store version – The version number if the data store is GDG.
    • Record format – Name of the record format.
  5. Press Apply to submit the job.
Parent topic: Work with Data Changer Jobs
Related information
Introduction
Create Data Changer Job Creation
Available Operations


Create Data Changer Job Creation

The Create Data Changer job executes a process that creates a JCL (the JCL is the base language for batch scripts in MVS, OS/390, or z/OS environment) that will execute the data changer process.

To create a Create Data Changer job:

  1. From the List of Jobs section in the Work with Jobs window, select Create Data Changer Jobs.
  2. In the General section, select the appropriate Machine ID and Company name.
  3. In the Data Store Qualification section, specify values for the following fields:
    • Application ID – Application ID, if applicable.
    • DB2 Owner, DL/I DBD – Owner of DB2 or DL/I DBD data store where data will be masked.
    • Data store type – Type of data store (ADA, VSAM, DB2, DL/I, GDG, SEQ, or ORA).
    • Data store name – Name of the data store where data will be masked.
    • Data store version – Version number if the data store is GDG.
    • Record format – Name of the record format.
  4. Press the Apply button. The Secondary Options window appears:
    Figure 1. BDCGENR – Create Data Masking Jobs

    BDCGENR – Create Data Masking Jobs

  5. In the Secondary Options window, specify values for the following fields:
    • JCL Output Data Set – Name of the data set containing the JCL. This field is mandatory.
    • JCL Output Member – Name of the member containing the JCL. This field is mandatory.
    • Temporary Data Set Prefix – Prefix of the required temporary data sets. This field is mandatory. The prefix must have a minimum of two qualifiers.
    • Submit Job – Indicator of whether or not the created JCL will be submitted.
Parent topic: Work with Data Changer Jobs
Related information
Introduction
Data Store Data Changer Job Creation
Available Operations


Available Operations

The buttons in the List of Scheduled Jobs section of the Work with Jobs window and the Secondary Options window are:

  • Change – Lets you modify File Qualification parameters for the selected job if the job is scheduled but has not yet been submitted (Flag Status is 0). Changes should be saved using the Apply button.
  • Apply – Saves the information for the new job, but keeps the window open.
  • New – Initializes fields for insertion when scheduling a new job.
  • Delete – Cancels the execution of the selected scheduled job and removes the job from the List of Scheduled Jobs list. If the job is currently being executed (Flag Status is 1), you will be prompted to confirm the deletion.
  • Cancel – Cancels any changes. Any confirmations made with the Apply button cannot be undone with the Cancel button but must be instead entered and confirmed again.
  • OK – Applies all changes and closes the window.
  • Reset All – Clears the values of all fields.
Parent topic: Work with Data Changer Jobs
Related information
Introduction
Data Store Data Changer Job Creation
Create Data Changer Job Creation


Distributed Exporter

The Distributed Exporter is a utility that creates and provides necessary information to the Extension Technology so that actual masking of your distributed data store can occur. This utility generates the following files that are used for masking after you have successfully exported from the Data Masking module:

  • method.rc – Coded content of source and target databases information.
  • elab.txt – List of data stores and the enabled columns within those data stores that need to be masked.
  • CREATETABLE.sql – list of the tables that need to be masked; a generic CREATE TABLE statement is provided for each table.
  • CREATEINDEX.sql – list of all the indexes for the tables that need to be masked; a generic CREATE INDEX statement is provided for each index.
  • ALTERTABLE_RI.sql – list of all the primary and foreign keys for the tables that need to be masked; a generic ALTER TABLE statement is provided for appropriate primary and foreign keys.
  • There are two areas of interest when it comes to using the Distributed Exporter with distributed data stores.
  • If you use ODBC-enabled data stores, you can mask across all your data stores with one invocation of the ODBC Extension. This feature is not available with the Oracle Extension.
  • You also have the ability to mask within one distributed data store, as long as a distinct target schema name is provided.

To access the Distributed Exporter, click [image] from the main window of the Data Masking module.

  • Window Contents
  • Available Operations
Parent topic: Data Masking Guide


Window Contents

This section describes the fields listed in the Distributed Exporter window as shown below:

Figure 1. Distributed Exporter

To export from the Data Masking module, you need to provide the following input:

  • your workspace (machine ID, company)
  • your source data store credentials (source database, source schema, source user ID/password)
    • For a data store that you will access using the ODBC Extension, the source database is an ODBC DSN configured to access that data store.
    • For a data store that you will access using the Oracle Extension, the source database is a BDE alias configured to access that data store. For more information, see Appendix Oracle Alias Creation in your Getting Started with Distributed Data Stores guide.
  • your target data store credentials (target database, target scheme, target user ID/password)
    • For a data store that you will access using the ODBC Extension, the target database is an ODBC DSN configured to access that data store.
    • For a data store that you will access using the Oracle Extension, the target database is an Oracle TNS service name configured to access that data store. For more information, see Appendix Oracle Alias Creation in your Getting Started with Distributed Data Stores guide.
  • directory where you want to place output files for Extension Technology (Output directory)
Note: If the source database and the target database are the same, the schemas cannot be the same.

In addition, if you are using ODBC-enabled data stores, you can provide multiple sets of source and target data store credentials.

Note: If you use the Distributed Exporter utility on one machine, and do your masking on another, make sure that the target ODBC DSN matches the ODBC DSN used for the actual masking. Similarly, if you use the Oracle Extension, make sure your Oracle TNS service names match.
Parent topic: Distributed Exporter
Related information
Available Operations


Available Operations

The buttons at the bottom of the Distributed Exporter window are:

  • Export – Exports all data store information to the specified output directory.
  • Close – Closes the window.
Parent topic: Distributed Exporter
Related information
Window Contents


Exit Routines

Within Data Masking, a series of exit routines are available to manage file migration. The available programs are prototypes that can be customized for the management of your database.

Note: A user-defined exit routine definition must comply with the linkage parameters associated with the routine type.
Tip: If you modify an exit routine, we recommend that you make copies of the programs and edit the copies rather than the original versions.

There are four main types of exit routines:

  • Writing
  • Discard
  • Field modification
  • Logging

Regarding each type of routine, a short outline of the predefined routines, their purposes and the ways to create new routines will be provided.

  • Writing Exit Routines
  • Discard Exit Routines
  • Modification Exit Routines
  • Logging Exit Routines
    A logging exit routine includes two programs aimed at managing the errors found during the masking process.
Parent topic: Data Masking Guide


Writing Exit Routines

A writing exit routine is a program that writes the migrated modified content of a sequential input file to a sequential output file.

The current migration procedure is a program written in C that performs the following tasks:

  • Opens the output file.
  • Closes the output file.
  • Given the modified record layout, writes to the output file.
  • Writing Routine Parameters
Parent topic: Exit Routines


Writing Routine Parameters

The linkage of the exit routine identified in the product PDS COPY by the kDCWRTW source includes the following fields.

01  kDCWRT-REC.
     05  kDCWRT-FLOPER              PIC  X(001).
     05  kDCWRT-LREC                PIC S9(009) COMP.
     05  kDCWRT-DSNAME.
           10  kDCWRT-DSNAME-52        PIC  X(056) VALUE SPACE.
           10                          PIC  X(001) VALUE LOW-VALUE.
     05  kDCWRT-RETCODE             PIC  X(002).
01  kDCWRT-RECORD                   PIC  X(16000).

These fields are described in the following table:

Field name | Input/Output | Description
kDCWRT-FLOPER | Input | Flag describing the action to be performed. Possible values: O - indicates that the call has been performed in the File opening mode C - indicates that the call has been performed in the File closing mode W - indicates that the call has been performed in the File writing mode
kDWRT-LREC | Input | Length of the record layout that will be written.
kDCWRT-DSNAME | Input | Name of the output sequential file.
kDCWRT-RETCODE | Output | Return code. If the return code is left BLANK, it means that the routine has been run successfully.
kDCWRT-RECORD | Output | File new record.
Parent topic: Writing Exit Routines


Discard Exit Routines

A discard exit routine is a program that skips writing records with certain characteristics during the migration phase.

  • Discard Routine Parameters
  • UDCUSSL Discard Exit Routine Example
Parent topic: Exit Routines


Discard Routine Parameters

The linkage of the exit routine identified in the product PDS COPY by the UDCUSSLW source includes the following fields:

01  UDCUSSL-RECORD                   PIC X(32760).
01  UDCUSSL-RETCODE                  PIC X(02).

These fields are described in the following table:

Field name | Input/Output | Description
UDCUSSL-RETCODE | Output | Return code. If the return code is left blank, it means that the routine has been performed successfully.
UDCUSSL-RECORD | Input | File record.
Parent topic: Discard Exit Routines


UDCUSSL Discard Exit Routine Example

In the following example of the UDCUSSL discard exit routine, a condition is tested. If this condition is not complied with, the return code is set to “ER”, thus disabling the writing of the record on the output file.

       IDENTIFICATION DIVISION.
       PROGRAM-ID.   UDCUSSL.
      ******************************************************************
      *
      *      Source name: UDCUSSL
      *      Function   : DATA CHANGER - USER EXIT FOR RECORD SELECTION
      *
      ******************************************************************
       ENVIRONMENT DIVISION.
       INPUT-OUTPUT SECTION.
       FILE-CONTROL.
       DATA DIVISION.
       FILE SECTION.
      ****************************************************************
       WORkING-STORAGE SECTION.
      ****************************************************************
      *
       01  WS-COMODI.
           05  FILLER                 PIC  X.
      *
       01  WS-AREA-RECORD.
           05  FILLER                 PIC  X(30).
           05  WS-FIELD-SELECT        PIC  X(10).
           05  FILLER                 PIC  X(82).
      ****************************************************************
       LINkAGE SECTION.
      ****************************************************************
       COPY UDCUSSLW.
      ****************************************************************
       PROCEDURE DIVISION  USING  UDCUSSL-RECORD UDCUSSL-RETCODE.
      ****************************************************************
      *
       START-PGM.
      *
           MOVE SPACE                     TO UDCUSSL-RETCODE.
      *
           MOVE UDCUSSL-RECORD(1:122)     TO WS-AREA-RECORD.
      *
           IF   WS-FIELD-SELECT        NOT = "CBL       "
                MOVE "ER"                 TO UDCUSSL-RETCODE
           END-IF.
      *
       END-PGM.
      *
           GOBACK.
      *
Parent topic: Discard Exit Routines


Modification Exit Routines

A modification exit routine includes a series of programs aimed at handling the information contained in the individual fields.

  • Predefined Modification Exit Routines List
  • Modification Exit Routines Parameters
Parent topic: Exit Routines
Related concepts
Appendix B: C Masking Routines for ODBC - Technical Description
Masking Routines


Predefined Modification Exit Routines List

Note: Unless otherwise stated, all routines are available for use with all data store types.

A list of predefined modification exit routines are described in the following table:

Routine Name | Description
BPMAADG | Sample routine for masking company name or surname according to data element content.
BPMNIC1 | Sample routine for changing dates.
BPMNICC | Generic alphanumeric code (can be used for telephone numbers). This routine changes numbers only, leaving all other characters untouched.
UDCADDC | Address masking routine.
UDCBLKC | Routine initializing the field with which it is associated. You can use it when you do not want some information to be displayed in the migrated data store.
UDCBPMK | Sample for masking complex codes.
UDCCDFC | Italian taxpayer code number masking routine.
UDCCFPI | Italian VAT number or taxpayer's code number masking routine. The routine associated with the mixed field containing both the VAT number and the taxpayer code number recognizes the content of the field and applies either one or the other routine.
UDCCHLC | Chilean tax code masking routine.
UDCCIFC | CIF (Spanish tax codes) masking routine.
UDCCOMC | Company name masking routine.
UDCCDCC | Credit card number masking routine.
UDCCDF1 | Sample routine for Italian tax code that considers date change.
UDCCFP1 | Sample routine for Italian tax code and VAT number that considers date change.
UDCEMAC | Email address masking routine.
UDCLOB1 | Template routine LOB/BLOB/CLOB columns containing positional data.
UDCNACC | Advanced name masking routine that preserves gender, and guarantees more distinct masked values than the standard name masking routine.
UDCNAMC | People's names masking routines to be used to mask personal detail files.
UDCNIFC | NIF (Spanish tax codes) masking routine.
UDCNUMC | Account number masking routine.
UDCPIVC | Italian VAT number masking routine.
UDCRANAG | Sample for forcing a fixed value in output.
UDCSNC | Surname and name masking routine.
UDCSNC2 | Surname and name masking routine specific to the Italian language.
UDCSNR2C | Surname and name or company name (in the same field) masking routine specific to the Italian language.
UDCSUCC | Advanced surname masking routine that guarantees more distinct masked values than the standard surname masking routine.
UDCSURC | Surname masking routine.
UDCTELC | Telephone number masking routine.
UDCUCST | String masking routine. It inverts the characters and then transcodes them according to a transcoding table.
UDCUSSN | Masking routine for USA Social Security Number.
UDCXML0 | Template routine for XML or LOB/BLOB/CLOB columns not containing data to be masked.
UDCXML1 | Template routine for XML or LOB/BLOB/CLOB columns containing XML data.
Parent topic: Modification Exit Routines


Modification Exit Routines Parameters

The linkage of the exit-routine identified in the product PDS COPY by the UDCUSFLW source includes the following fields:

01  UR-ACTION                             PIC  X(04).
01  UR-FILE-INFO.
    03 UR-FILE-NAME                     PIC  X(128).
    03 UR-FILE-TYPE                       PIC  X(04).
01  UR-RECORD                             PIC  X(32760).
01  UR-FIELD.
    03 UR-INP-VAL                            PIC  X(256).
    03 UR-POS                                     PIC S9(05) COMP.
    03 UR-LEN                                     PIC S9(05) COMP.
    03 UR-TYPE                                   PIC  X(01).
    03 UR-INT                                      PIC S9(03) COMP.
    03 UR-DEC                                     PIC S9(03) COMP.
    03 UR-SIGN                                   PIC  X(01).
    03 UR-NULL                                  PIC  X(01).
    03 UR-VARCHAR                         PIC  X(01).
    03 UR-INULL-VAL                       PIC  X(01).
       88 UR-VAL-NULL     VALUE  X"6F".
    03 UR-OTHER-FIELDS-NBR       PIC S9(04) COMP.
01  UR-OTHER-FIELDS.
    03 OCCURS 50.
       05 UR-P-FILE-NAME                PIC  X(128).
       05 UR-P-FILE-TYPE                  PIC  X(04).
       05 UR-P-INP-VAL                      PIC  X(256).
       05 UR-P-POS                               PIC S9(05) COMP.
       05 UR-P-LEN                               PIC S9(05) COMP.
       05 UR-P-TYPE                             PIC  X(01).
       05 UR-P-INT                                PIC S9(03) COMP.
       05 UR-P-DEC                               PIC S9(03) COMP.
       05 UR-P-SIGN                              PIC  X(01).
       05 UR-P-NULL                             PIC  X(01).
       05 UR-P-VARCHAR                    PIC  X(01).
       05 UR-P-INULL-VAL                  PIC  X(01).
          88 UR-P-VAL-NULL    VALUE  X"6F".
01 UR-OUTPUT.
    03 UR-OUT-VAL                             PIC  X(256).
    03 UR-OUT-FLDTYPE                    PIC  X(01).
    03 UR-OUT-FLDLEN                      PIC S9(03) COMP.
    03 UR-OUT-FLDINT                       PIC S9(03) COMP.
    03 UR-OUT-FLDDEC                      PIC S9(03) COMP.
    03 UR-OUT-FLDSIGN                     PIC  X(01).
    03 UR-RETCODE                            PIC  X(02).
    03 UR-MSG-CODE                         PIC  X(07).
    03 UR-MSG-PARM                         PIC  X(128).
    03 UR-OUT-TYPE                           PIC  X(01).
    03 UR-DESCR                                  PIC  X(80).

These fields are described in the following table:

Field name | Input/Output | Description
UR-ACTION | Input | Flag describing the action to be performed. Possible values: INIT - (not used with this product but included for future use) indicates that the call processes a record read from an original file. LAST - indicates that the call has been made at the end of the record processing operation. It is only available to routines customized by the user. AUTH - (not used with this product but included for future use) indicates that the call returns a string containing the author of the exit routine in UR-DESCR VERS - (not used with this product but included for future use) indicates that the call returns a string containing the version of the exit routine in UR-DESCR INFO - (not used with this product but included for future use) indicates that the call returns a string containing the title of the exit routine in UR-DESCR DESC - (not used with this product but included for future use) indicates that the call returns, in UR-DESCR, a row in a table containing the description of the exit routine INPT - (not used with this product but included for future use) indicates that the call returns, in UR-DESCR, a row in a table containing: string 1 (to indicate that the user exit has only one input parameter), the type and the length of the input field OUTP - (not used with this product but included for future use) indicates that the call returns, in UR-DESCR, a row in a table containing: the type and the length of the output field
UR-FILE-NAME | Input | Not used
UR-FILE-TYPE | Input | Not used
UR-RECORD | Input | File Record
UR-INP-VAL | Input | Value contained in the field to be migrated.
UR-POS | Input | Position of the field to be migrated inside the record.
UR-LEN | Input | Length of the field to be migrated.
UR-TYPE | Input | Type of the field to be migrated.
UR-INT | Input | Number of integers (for numeric fields only).
UR-DEC | Input | Number of decimals (for numeric fields only).
UR-SIGN | Input | Sign of the field to be migrated.
UR-NULL | Input | Content of any null byte (MVS only).
UR-VARCHAR | Input | Length of the VARCHAR field (MVS only).
UR-INULL-VAL | Input | Byte value possibly containing the null value (MVS only).
UR-OTHER-FIELDS-NBR | Input | Not used
UR-OTHER-FIELDS | Input | Source encoding type in its UR-P-FILE-NAME field
UR-OUT-VAL | Output | Value contained in the migrated field
UR-OUT-FLDTYPE | Output | Not used
UR-OUT-FLDLEN | Output | Length of the masked value.
UR-OUT-FLDINT | Output | Not used
UR-OUT-FLDDEC | Output | Not used
UR-OUT-FLDSIGN | Output | Not used
UR-RETCODE | Output | Return code (Space = OK)
UR-MSG-CODE | Output | Not used
UR-MSG-PARM | Output | Not used
UR-OUT-TYPE | Output | Flag describing the location of the processed value. Possible flags: R - The change worked directly on UR-RECORD F - The output value is in UR-OUT-VAL
UR-DESCR | Output | (Not used now, but included for future use.) It contains additional information about the exit routine.
Parent topic: Modification Exit Routines


Logging Exit Routines

A logging exit routine includes two programs aimed at managing the errors found during the masking process.
  • Predefined Masking and Logging Exit Routines List
  • Masking Routine UDCMASPG - Prototype
  • Masking Routine UDCLOGPG – Prototype
  • Logging Table HSLOGTAB
Parent topic: Exit Routines


Predefined Masking and Logging Exit Routines List

Note: Unless otherwise stated, all routines are available for use with all data store types.

A list of predefined logging exit routines is described in the following table:

Routine Name | Description
UDCMASPG | Generic routine for masking IBAN or Credit Card (MVS only).
UDCLOGPG | Manage logging messages.
Parent topic: Logging Exit Routines


Masking Routine UDCMASPG - Prototype

The Data Express routine UDCMASPG can now be used to change an existing customer’s masking routines. This routine contains new rules to follow in case of an error. Please see the following steps:

  • Move “ER” to linkage field UR-RETCODE.
  • Assign different values to field UR-MSG-CODE depending on the values used to populate the table HSLOGTAB.
  • Move the routine's name to field UR-MSG-PARM.

All of the information will be used in order to access HSLOGTAB and if there is a logging routine to be called, it will be called.

Parent topic: Logging Exit Routines


Masking Routine UDCLOGPG – Prototype

The Data Express routine UDCLOGPG contains an example of managing logging errors. The routine produces an output file managing the masking routine, name that finished in error, the type of error, the related descriptions, and the data store name.

The following example shows a row produced by the routine:

UDCMASPG FUNCTION ERROR THE STRING DOES NOT CONTAIN A VALID VALUE THE INPUT VALUE:FR1212345123451234567890112 TABLE:LOGGING

Note: In order to correctly manage the prototype routine provided (UDCLOGPG), it is advisable to add into JCLs BDCCHGR and BTESBMR the previous JCL card in order to work correctly with the Logging file produced by that routine, according to the way you choose to develop your Logging routine.

The JCL’s card showed below is only an example to follow in case you decide to write an output file to log the masking’s errors as is performed into UDCLOGOPG:

//LOGSRC DD DISP=(NEW,CATLG,CATLG),
//          DSN=LOGGING OTUPUT FILE NAME,
//          DCB=(RECFM=FB,LRECL=327),
//          SPACE=(CYL,(10,10))
Parent topic: Logging Exit Routines


Logging Table HSLOGTAB

This table contains the information to call the logging routine depending on the Msgcode used in the masking routine.

For more information about this table, see the Table and column description in the Data Model Guide.

In order to populate this table, please follow the example below:

INSERT INTO tableowner.HSLOGTAB (MSGCODE, FLAGENVI, ROUTNAME, VALRCODE)
VALUES ('LCI0001', 'Y', 'UDCLOGPG', '00');

The population of the table is very important since it allows associating one different logging routine for each MSGCODE or associating the same logging routine for all MSGCODE inserted into the table.

These differences provide choices between creating and managing of one routine and creating and managing several routines

Parent topic: Logging Exit Routines


Appendix A: COBOL Masking Routines Technical Description

This appendix provides a technical description of how masking routines work; particular detail is given to the meaning of the fields within the Data Masking interface.

  • A.1. Masking Routines Interface
  • A.2. Interface Source
  • A.3. Interface Description
  • A.4. UR-ACTION Values
Parent topic: Data Masking Guide


A.1. Masking Routines Interface

All masking routines have the same interface. This interface is described, in the product, with a COBOL copybook named UTEUSFLW.

Parent topic: Appendix A: COBOL Masking Routines Technical Description
Related information
A.2. Interface Source
A.3. Interface Description
A.4. UR-ACTION Values


A.2. Interface Source

  • A.2.1. Data Express for z/OS
  • A.2.2. Data Express for Distributed Systems
Parent topic: Appendix A: COBOL Masking Routines Technical Description
Related information
A.1. Masking Routines Interface
A.3. Interface Description
A.4. UR-ACTION Values


A.2.1. Data Express for z/OS

      01 UR-ACTION                   PIC  X(32760).
      01 UR-FILE-INFO.
         03 UR-FILE-NAME             PIC  X(44).
         03 UR-FILE-TYPE             PIC  X(04).

      01 UR-RECORD                   PIC  X(32760).

      01 UR-FIELD.
          03 UR-INP-VAL               PIC  X(256).
          03 UR-POS                   PIC S9(05) COMP.
          03 UR-LEN                   PIC S9(05) COMP.
          03 UR-TYPE                  PIC  X(01).
          03 UR-INT                   PIC S9(03) COMP.
          03 UR-DEC                   PIC S9(03) COMP.
          03 UR-SIGN                  PIC  X(01).
          03 UR-NULL                  PIC  X(01).
          03 UR-VARCHAR               PIC  X(01).
          03 UR-INULL-VAL             PIC  X(01).
             88 UR-VAL-NULL           VALUE  X"6F".
          03 UR-OTHER-FIELDS-NBR      PIC S9(04) COMP.

      01 UR-OTHER-FIELDS.
          03 OCCURS 50.
             05 UR-P-FILE-NAME        PIC  X(44).
             05 UR-P-FILE-TYPE        PIC  X(04).
             05 UR-P-INP-VAL          PIC  X(256).
             05 UR-P-POS              PIC S9(05) COMP.
             05 UR-P-LEN              PIC S9(05) COMP.
             05 UR-P-TYPE             PIC  X(01).
             05 UR-P-INT              PIC S9(03) COMP.
             05 UR-P-DEC              PIC S9(03) COMP.
             05 UR-P-SIGN             PIC  X(01).
             05 UR-P-NULL             PIC  X(01).
             05 UR-P-VARCHAR          PIC  X(01).
             05 UR-P-INULL-VAL        PIC  X(01).
                88 UR-P-VAL-NULL      VALUE  X"6F".

      01 UR-OUTPUT.
          03 UR-OUT-VAL               PIC  X(256).
          03 UR-OUT-FLDTYPE           PIC  X(01).
          03 UR-OUT-FLDLEN            PIC S9(03) COMP.
          03 UR-OUT-FLDINT            PIC S9(03) COMP.
          03 UR-OUT-FLDDEC            PIC S9(03) COMP.
          03 UR-OUT-FLDSIGN           PIC  X(01).
     *
          03 UR-RETCODE               PIC  X(02).
     *
          03 UR-MSG-CODE              PIC  X(07).
          03 UR-MSG-PARM              PIC  X(128).
          03 UR-OUT-TYPE              PIC  X(01).
     *
          03 UR-DESCR                 PIC  X(80).
Parent topic: A.2. Interface Source


A.2.2. Data Express for Distributed Systems

01  UR-ACTION                   PIC  X(04).

       01  UR-FILE-INFO.
           03 UR-FILE-NAME             PIC  X(44).
           03 UR-FILE-TYPE             PIC  X(04).

       01  UR-RECORD                   PIC  X(32760).

       01  UR-FIELD.
           03 UR-INP-VAL               PIC  X(256).
           03 UR-POS                   PIC  S9(9) COMP-5.
           03 UR-LEN                   PIC  S9(9) COMP-5.
           03 UR-TYPE                  PIC  X(01).
           03 UR-INT                   PIC  S9(9) COMP-5.
           03 UR-DEC                   PIC  S9(9) COMP-5.
           03 UR-SIGN                  PIC  X(01).
           03 UR-NULL                  PIC  X(01).
           03 UR-VARCHAR               PIC  X(01).
           03 UR-INULL-VAL             PIC  X(01).
              88 UR-VAL-NULL           VALUE  X"6F".
           03 UR-OTHER-FIELDS-NBR      PIC  S9(4) COMP-5.

       01  UR-OTHER-FIELDS.
           03 OCCURS 50.
              05 UR-P-FILE-NAME        PIC  X(44).
              05 UR-P-FILE-TYPE        PIC  X(04).
              05 UR-P-INP-VAL          PIC  X(256).
              05 UR-P-POS              PIC  S9(9) COMP-5.
              05 UR-P-LEN              PIC  S9(9) COMP-5.
              05 UR-P-TYPE             PIC  X(01).
              05 UR-P-INT              PIC  S9(9) COMP-5.
              05 UR-P-DEC              PIC  S9(9) COMP-5.
              05 UR-P-SIGN             PIC  X(01).
              05 UR-P-NULL             PIC  X(01).
              05 UR-P-VARCHAR          PIC  X(01).
              05 UR-P-INULL-VAL        PIC  X(01).
                 88 UR-P-VAL-NULL      VALUE  X"6F".

       01 UR-OUTPUT.
           03 UR-OUT-VAL               PIC  X(256).
           03 UR-OUT-FLDTYPE           PIC  X(01).
           03 UR-OUT-FLDLEN            PIC  S9(9) COMP-5.
           03 UR-OUT-FLDINT            PIC  S9(9) COMP-5.
           03 UR-OUT-FLDDEC            PIC  S9(9) COMP-5.
           03 UR-OUT-FLDSIGN           PIC  X(01).
      *
           03 UR-RETCODE               PIC  X(02).
      *
           03 UR-MSG-CODE              PIC  X(08).
           03 UR-MSG-PARM              PIC  X(128).
           03 UR-OUT-TYPE              PIC  X(01).
      *
           03 UR-DESCR                 PIC  X(80).
Parent topic: A.2. Interface Source


A.3. Interface Description

Field | Type | Meaning
UR-ACTION | Input | Type of operation (see Values of UR-ACTION table).
UR-FILE-NAME | Unused | For future use.
UR-RECORD | Input-Output | The first 32,760 bytes (left-aligned and padded with spaces) of the read record, in the same format as a sequential unload.
UR-INP-VAL | Input | The field to be masked.
UR-POS | Input | The position of the field to be masked, in the file layout.
UR-LEN | Input | The length of the field to be masked.
UR-TYPE | Input | The type of the field to be masked. Options are: A - Alphanumeric P - Packed N - Numeric zoned B - Binary
UR-INT | Input | The integer number of the field to be masked.
UR-DEC | Input | The decimal number of the field to be masked.
UR-SIGN | Input | The sign of the field to be masked. Options are: Y - Signed N - Unsigned space - Non-numeric field
UR-NULL | Input | The null-capability of the field to be masked. Options are: Y - Null capable space - Not null capable
UR-VARCHAR | Input | The variable length property of the field to be masked. Options are: Y - Variable length space - Not variable length
UR-INULL-VAL | Input | If a column accepts NULL values and it is actually NULL, then the UR-INULL-VAL field is set to "o" (6f), if it is different than NULL, the UR-INULL-VAL is blank " ". The "?" sign means that the column is NULLABLE and the value is NULL. If a column accepts NULL values, then after it is set in the UR-RECORD, we add a byte with the "?".
UR-OTHER-FIELDS-NBR | Unused | For future use.
UR-OTHER-FIELDS | Unused | Source encoding type in its UR-P-FILE-NAME field
UR-OUT-VAL | Output | The value of the masked field, if UR-OUT-TYPE is equal to F .
UR-OUT-FLDTYPE | Unused | For future use.
UR-OUT-FLDLEN | Output | Output length of the masked value. Data Express for z/OS initializes this field value to the actual length of UR-INP-VAL, but you can explicitly set the value for UR-OUT-FLDLEN.
UR-FLDINT | Unused | For future use.
UR-FLDDEC | Unused | For future use.
UR-FLDSIGN | Unused | For future use.
UR-RETCODE | Output | Routine return code. Options are: space character(s) -masking routine works as expected. 01 or 02 - a problem was encountered and the JCL return code is set to 8. If the UR-RETCODE value is 01 or 02 and UR-OUT-TYPE is equal to F, the field is not masked.
UR-MSG-CODE | Unused | For future use.
UR-MSG-PARM | Unused | For future use.
UR-OUT-TYPE | Output | Flag describing the location of the processed value. Possible flags: F - The output value is in UR-OUT-VAL R - The change worked directly on UR-RECORD
UR-DESCR | Unused | For future use.
Parent topic: Appendix A: COBOL Masking Routines Technical Description
Related information
A.1. Masking Routines Interface
A.2. Interface Source
A.4. UR-ACTION Values


A.4. UR-ACTION Values

Value(s) | Meaning
INIT | The routine is called with this value at the beginning of each file elaboration. The code executed in this case can be an initialization code. The predefined routines set the return code to a space character.
EXEC | The routine is called with this value for each read record. This is the masking routine.
LAST | The routine is called with this value at the end of each file elaboration. The code executed in this case can be a final elaboration code. The predefined routines do not use this value because no final elaboration is needed.
AUTH , VERS , INFO , DESC , INPT , OUTPT | The routine is never called for these values of the parameter. The code included in the predefined routines is written for documentation purposes. We suggest you write similar code for customized routines.
Parent topic: Appendix A: COBOL Masking Routines Technical Description
Related information
A.1. Masking Routines Interface
A.2. Interface Source
A.3. Interface Description


Appendix B: C Masking Routines for ODBC - Technical Description

This appendix provides a technical description of how C masking routines for ODBC work; particular detail is given to the meaning of the fields within the Data Masking interface.

For ODBC extension, Data Express provides two types of masking routines: standard masking and extended masking. Standard masking routines have limited capabilities as opposed to extended masking routines, meaning that they mask only one value, are not based on dependencies, and use only the source value and its length. Also, when two or more source values are related, the masking of one could depend on the other.
Note: This limitation does not apply to the Oracle extension because its masking routines exist as part of a working chain rather than within a DLL. Hence, the routine itself has information about all columns.

The extended masking routines, provided in Data Express starting with Update 17, have the same interface as the COBOL masking routine.

Interface Details

The interface for the extended C masking routines allows you to decide exactly what kind of masking to perform, based on additional parameters including:
UR-RECORD
String-based variable that contains all record values.
Source value parameters
A structure, containing all metadata of the source value like type, length, int, dec, position (the starting position in the UR-RECORD), including the value itself.
Output values structure
A structure that stores the output value and the masking type, which can be either by a single masked value or extracted from the UR-RECORD.

For complete information, see Masking Routines in Getting Started with Distributed Data Stores

Examples

The following examples show the complexity of extended masking as opposed to standard masking:

Standard Masking
char* UDCNAMC(char *szValue, unsigned int *nLength)
Extended Masking
char * UDCNAMC(char *action, char *fileinfo, char *record,
   struct fldstruct *chgparams_fstruct, struct fldodata *chgparams_odata,
   struct fldoutput *chgparams_out)
action
INIT, EXEC, or LAST. For more information, see Modification Exit Routines Parameters in your Data Masking Guide.
fileinfo
Used only in z/OS.
record
UR-RECORD
fldstruct
The structure containing the input source information.
struct fldstruct
    {
        unsigned char f_inpval[UR_INP_VAL_LEN];
        int f_posn;
        int f_len;
        unsigned char f_fldtype;
        short f_inttype;
        short f_dectype;
        unsigned char f_sign;
        unsigned char f_null;
        unsigned char f_varchar;
        unsigned char f_inull;
        short f_otherfld;
    } ;
f_inpval
The source value.
f_posn
Its position in the UR-RECORD.
f_len
The column length.
f_fldtype
The column type.
f_inttype
int type.
f_dectype
dec type.
f_sign
Whether value is signed, or unsigned.
f_null
Whether column is nullable, or not.
f_varchar
Whether varchar is signed, or unsigned.
f_inull
Whether the value is NULL, or not.
f_otherfld
Used only in z/OS.
fldodata
Unused structure in the distributed engine.
struct fldodata
    {
        unsigned char oth_filename[UR_FILE_NAME_LEN];
        unsigned char oth_filetype[4];
        unsigned char oth_inpval[UR_INP_VAL_LEN];
        short oth_posn; /*short*/
        short oth_len; /*short*/
        unsigned char oth_fldtype;
        short oth_inttype; /*short*/
        short oth_dectype; /*short*/
        unsigned char oth_sign;
        unsigned char oth_null;
        unsigned char oth_varchar;
        unsigned char oth_inull;
        short oth_otherfld;
    } ;
fldoutput
Output structure that is being returned to the DE engine.
struct fldoutput
    {
        char out_outval[UR_INP_VAL_LEN];
        char out_fldtype;
        short out_fldlen; /*short*/
        short out_inttype; /*short*/
        short out_dectype; /*short*/
        char out_fldsign;
        char out_rtncode[2];
        char out_msgcode[7]; /*7*/
        char out_msgparm[UR_OUT_MSGPARM_LEN];
        char out_type;
        char out_descr[80];
    };
out_outval
The masked value.
out_fldtype
The output column type.
out_fldlen
The output column length.
out_inttype
The output int type.
out_dectype
The output dec type.
out_fldsign
The output value sign.
out_rtncode
The return code of the masking routine. If the masking is successful, the return code is " " (space).
out_msgcode
Used only in z/OS
out_msgparm
Used only in z/OS
out_type
Identifies the output type. One of:
R | The out_outval value is ignored and what is in the UR-RECORD is inserted in the target environment.
F | The masked value used is the one stored in out_outval.
out_desrc
Used only in z/OS
*
#define UR_INP_VAL_LEN	256
#define UR_RECORD_LEN	32760
#define UR_FILE_NAME_LEN	128
#define UR_OUT_MSGPARM_LEN 128
Parent topic: Data Masking Guide
Related concepts
Masking Routines
Modification Exit Routines


Appendix C: C Masking Routines for OCI - Technical Description

This appendix provides a technical description of how C masking routines for OCI work.

Oracle OCI extension of Data Express works as a chain of processes that passes information about each targeted column, such as value, type, length, precision, scale and so on. In contrast to ODBC masking routines, C masking routines for OCI are built as separate processes, and are not included in a DLL. Thus, each masking process (masking routine) becomes a part of the main Data Express processing chain, and can use all the information it passes.

Functional Analysis
C masking routines provide additional functionality over ODBC masking routines in that each routine can:
  • Access and use all of the data related to a column
  • Modify data values on the fly
  • Pass all changed data back to Data Express for insertion into the target
Technical Analysis
The Data Express OCI extension uses predefined data structures that hold column information. It also uses functions that read and write the metadata passed to and from each part of the processing chain. We also provide these functions in a library file so that you can use them in your own written routines just as Data Express uses them.
Parent topic: Data Masking Guide


Data Model Guide

Provides description of the table structures used in the Data Builder, Data Masking, and Data Subset Extraction modules.
  • Table and Column Descriptions
    Describes the table and columns in the product.
  • Sequential Files
    Describes information for working with sequential files.
Parent topic: Data Express


Table and Column Descriptions

Data Express updates tables contained in the DB2 or Windows Knowledge Base during product installation. This chapter provides a full description of the tables and columns.

  • ANDCHFLF
    Describes the ANDCHFLF table and columns.
  • ANENVDLS
    Describes the ANENVDLS table and columns.
  • ANENVLST
    Describes the ANENVLST table and columns
  • ANLICCHG
    Describes the ANLICCHG table and columns.
  • ANLICLOG
    Describes the ANLICLOG table and columns.
  • ANURDCLA
    Describes the ANURDCLA table and columns.
  • ANURDFLC
    Describes the ANURDFLC table and columns.
  • ANURDFLF
    Describes the ANURDFLF table and columns.
  • ANURDFLN
    Describes the ANURDFLN table and columns.
  • ANURDFXC
    Describes the ANURDFXC table and columns.
  • ANURDFXF
    Describes the ANURDFXF table and columns.
  • ANURDIMC
    Describes the ANURDIMC table and columns.
  • ANURDIMG
    Describes the ANURDIMG table and columns.
  • ANURDIXC
    Descibes the ANURDIXC table and columns.
  • ANURDLOG
    Describes the ANURDLOG table and columns.
  • ANURDRFC
    Describes the ANURDRFC table and columns.
  • ANURDSNC
    Describes the ANURDSNC table and columns.
  • ANURDSPE
    Describes the ANURDSPE table and columns.
  • ANURDSPN
    Describes the ANURDSPN table and columns.
  • ANURDSTR
    Describes the ANURDSTR table and columns.
  • DDDTMCOL
    Describes the DDDTMCOL table and columns.
  • DDDTMDEC
    Describes the DDDTMDEC table and columns.
  • DDDTMREF
    Describes the DDDTMREF table and columns.
  • DDDTMSTQ
    Describes the DDDTMSTQ table and columns.
  • DDDTMTAG
    Describes the DDDTMTAG table and columns.
  • HSDCHCLA
    Describes the HSDCHCLA table and columns.
  • HSDCHFIL
    Describes the HSDCHFIL table and columns.
  • HSDGNCOD
    Describes the HSDGNCOD table and columns.
  • HSDGNDIR
    Describes the HSDGNDIR tables and columns.
  • HSDGNDIZ
    Describes the HSDGNDIZ table and columns.
  • HSDGNFIL
    Describes the HSDGNFIL table and columns.
  • HSDGNFLT
    Describes the HSDGNFLT table and columns.
  • HSDGNLAW
    Describes the HSDGNLAW tables and columns.
  • HSENVELB
    Describes the HSENVELB table and columns.
  • HSENVEXT
    Describes the HSENVEXT table and columns.
  • HSENVFLT
    Describes the HSENVFLT table and columns.
  • HSENVGRP
    Describes the HSENVGRP table and columns.
  • HSENVLMT
    The HSENVLMT table is used for storing the cardinality limit of records produced during the subsetting process. It is populated when using Work with Method and Work with Company phases:
  • HSENVMTH
    Describes the HSENVMTH table and columns.
  • HSENVPAR
    Describes the HSENVPAR table and columns.
  • HSENVPRN
    Describes the HSENVPRN table and columns.
  • HSENVRRD
    Describes the HSENVRRD table and columns.
  • HSENVRRH
    Describes the HSENVRRH table and columns.
  • HSENVSTP
    Describes the HSENVSTP table and columns.
  • HSLICCHG
    Describes the HSLICCHG table and columns.
  • HSLICFIL
    Describes the HSLICFIL table and columns.
  • HSLICFLD
    Describes the HSLICFLD table and columns.
  • HSLICFTF
    Describes the HSLICFTF table and columns.
  • HSLICLID
    Describes the HSLICLID table and columns.
  • HSLICSCD
    Describes the HSLICSCD table and columns.
  • HSLICSTR
    Describes the HSLICSTR table and columns.
  • HSLOGTAB
    Describes the HSLOGTAB table and columns.
  • HSSYSAUX
    Describes the HSSYSAUX table and columns.
  • HSSYSCKS
    Describes the HSSYSCKS table and columns.
  • HSSYSCOL
    Describes the HSSYSCOL table and columns.
  • HSSYSDAU
    Describes the HSSYSDAU table and columns.
  • HSSYSDBA
    Describes the HSSYSDBA table and columns.
  • HSSYSDEP
    Describes the HSSYSDEP table and columns.
  • HSSYSFKE
    Describes the HSSYSFKE table and columns.
  • HSSYSIND
    Describes the HSSYSIND table and columns.
  • HSSYSIPA
    Describes the HSSYSIPA table and columns.
  • HSSYSKEY
    Describes the HSSYSKEY table and columns.
  • HSSYSRAU
    Describes the HSSYSRAU table and columns.
  • HSSYSREL
    Describes the HSSYSREL table and columns.
  • HSSYSSYN
    Describes the HSSYSSYN table and columns.
  • HSSYSTAB
    Describes the HSSYSTAB table and columns.
  • HSSYSTAU
    Describes the HSSYSTAU table and columns.
  • HSSYSTPA
    Describes the HSSYSTPA table and columns.
  • HSSYSTSP
    Describes the HSSYSTSP table and columns.
  • HSSYSVDE
    Describes the HSSYSVDE table and columns.
  • HSSYSVIE
    Describes the HSSYSVIE table and views.
  • HSURDAPP
    Describes the HSURDAPP table and columns.
  • HSURDAUT
    The HSURDAUT table is used for storing user profiling information, and is populated during the User Credentials phase:
  • HSURDCAR
    Describes the HSURDCAR table and columns.
  • HSURDCJD
    Describes the HSURDCJD table and columns.
  • HSURDCJH
    Describes the HSURDCJH table and columns.
  • HSURDCLA
    Describes the HSURDCLA table and columns.
  • HSURDCOL
    Describes the HSURDCOL table and columns.
  • HSURDCOM
    Describes the HSURDCOM table and columns.
  • HSURDCPY
    Describes the HSURDCPY table and columns.
  • HSURDDFT
    Describes the HSURDDFT table and columns.
  • HSURDEKY
    Describes the HSURDEKY table and columns.
  • HSURDEXT
    Describes the HSURDEXT table and columns.
  • HSURDFIL
    Describes the HSURDFIL table and columns.
  • HSURDFIN
    Describes the HSURDFIN table and columns.
  • HSURDINT
    Describes the HSURDINT table and columns.
  • HSURDICP
    Describes the HSURDICP table and columns.
  • HSURDJOB
    Describes the HSURDJOB table and columns.
  • HSURDKEY
    Describes the HSURDKEY table and columns.
  • HSURDLIC
    Describes the HSURDLIC table and columns.
  • HSURDLOG
    Describes the HSURDLOG table and columns.
  • HSURDMF
    Describes the HSURDMF table and columns.
  • HSURDMFD
    Describes the HSURDMFD table and columns.
  • HSURDMID
    Describes the HSURDMID table and columns.
  • HSURDNCC
    Describes the HSURDNCC table and columns.
  • HSURDPID
    Describes the HSURDPID table and columns.
  • HSURDPRG
    Describes the HSURDPRG table and columns.
  • HSURDPRO
    Describes the HSURDPRO table and columns.
  • HSURDROU
    Describes the HSURDROU table and columns.
  • HSURDSYC
    Describes the HSURDSYC table and columns.
  • HSURDSYT
    Describes the HSURDSYT table and columns.
  • HSURDUE
    Describes the HSURDUE table and columns.
  • HSURDUEV
    Describes the HSURDUEV table and columns.
  • HSURDUSR
    The HSURDUSR table is used for storing user profiling information, and is populated during the User Credentials phase:
  • MSURDLOG
    Describes the MSURDLOG table and columns.
  • USDCHADD
    Describes the USDCHADD table and columns.
  • USDCHCOM
    Describes the USDCHCOM table and columns.
  • USDCHEMA
    Describes the USDCHEMA table and columns.
  • USDCHNAC
    Describes the USDCHNAC table and columns.
  • USDCHNAM
    Describes the USDCHNAM table and columns.
  • USDCHSUC
    Describes the USDCHSUC table and columns.
  • USDCHSUR
    Describes the USDCHSUR table and columns.
  • USURDCBE
    Describes the USURDCBE table and columns.
  • USURDCBZ
    Describes the USURDCBZ table and columns.
  • USURDDBM
    Describes the USURDDBM table and columns.
  • USURDFLC
    Describes the USURDFLC table and columns.
  • USURDIMP
    Describes the USURDIMP table and columns.
  • USURDRIR
    Describes the USURDRIR table and columns.
  • V1DCHFIL
    Describes the V1DCHFIL table and columns.
  • V1DCHFLF
    Describes the V1DCHFLF table and columns.
  • V1REP01
    Describes the V1REP01 table and columns.
  • V1REP02
    Describes the V1REP02 table and columns.
  • V1REP03
    Describes the V1REP03 table and columns.
  • V1REP04
    Describes the V1REP04 table and columns.
  • V1REP05
    Describes the V1REP05 table and columns.
  • V1REP06
    Describes the V1REP06 table and columns.
  • V1REP07
    Describes the V1REP07 table and columns.
  • V1REP08
    Describes the V1REP08 table and columns.
  • V1REP09
    Describes the V1REP09 table and columns.
  • V1REP10
    Describes the V1REP10 table and columns.
  • V1REP11
    Describes the V1REP11 table and columns.
  • V1REP12
    Describes the V1REP12 table and columns.
  • V1REP13
    Describes the V1REP13 table and columns.
  • V1REP14
    Describes the V1REP14 table and columns.
  • V1REP15
    Describes the V1REP15 table and columns.
  • V1URDCPY
    Describes the V1URDCPY table and columns.
  • V1URDFIL
    Describes the V1URDFIL table and columns.
  • V1URDFLF
    Describes the V1URDFLF table and columns.
  • V1URDIMC
    Describes the V1URDIMC table and columns.
  • V1URDIMG
    Describes the V1URDIMG table and columns.
  • V1URDLOG
    Describes the V1URDLOG table and columns.
  • V1URDRFC
    Describes the V1URDRFC table and columns.
  • V2URDFLF
    Describes the V2URDFLF table and columns.
  • V3URDFLF
    Describes the V3URDFLF table and columns.
  • XPURDFLC
    Describes the XPURDFLC table and columns.
Parent topic: Data Model Guide


ANDCHFLF

The ANDCHFLF table is used for data element extensions and is populated during the Data Changer phase. The following table describes the columns in the ANDCHFLF table:
Column | Type | Length | Decimal | Domain
MCRECID | BINARY | 4 | 0 | machine/company ID
FILRECID | BINARY | 9 | 0 | file ID
FLDRECID | BINARY | 9 | 0 | field ID
CHGFLG | CHARACTER | 1 | 0 | Data Changer active
CHGPGM | CHARACTER | 8 | 0 | Data Changer program
METHOD | CHARACTER | 10 | 0 | method
Parent topic: Table and Column Descriptions


ANENVDLS

The ANENVDLS table is used for synthetic data lists, and is populated during the Data Generation phase. The following table describes the columns in the ANENVDLS table:

Column | Type | Length | Decimal | Domain
METHOD | CHARACTER | 20 | 0 | code name
MCRECID | BINARY | 4 | 0 | machine/company ID
CLATYPE | BINARY | 4 | 0 | class identifier
FLTPRG | BINARY | 4 | 0 | class progressive
VALUE | CHARACTER | 20 | 0 | parameter value
STEP | BINARY | 4 | 0 | step
Parent topic: Table and Column Descriptions


ANENVLST

The ANENVLST table is used for test environment lists, and is populated during the Environment Creation phase. The following table describes the columns in the ANENVLST table:

Column | Type | Length | Decimal | Domain
METHOD | CHARACTER | 10 | 0 | method name
MCRECID | BINARY | 4 | 0 | machine/company ID
CLATYPE | BINARY | 4 | 0 | class identifier
FLTPRG | BINARY | 4 | 0 | filter prog
VALUE | CHARACTER | 230 | 0 | parameter value
STEP | BINARY | 4 | 0 | step
Parent topic: Table and Column Descriptions


ANLICCHG

The ANLICCHG table is used for applying changes and is populated during the Life Cycle phase. The following table describes the columns in the ANLICCHG table:

Column | Type | Length | Decimal | Domain
OLDMCRECID | BINARY | 4 | 0 | old machine/company ID
NEWMCRECID | BINARY | 4 | 0 | new machine/company ID
OLDFILRECID | BINARY | 9 | 0 | old file ID
NEWFILRECID | BINARY | 9 | 0 | new file ID
APPDATE | CHARACTER | 8 | 0 | application change date
Parent topic: Table and Column Descriptions


ANLICLOG

The ANLICLOG table is used for file fields, and is populated during the Life Cycle phase. The following table describes the columns in the ANLICLOG table:

Column | Type | Length | Decimal | Domain
MCRECID | SMALLINT | 2 | 0 | machine/company ID
FILRECID | INTEGER | 4 | 0 | file ID
FILLIB | CHAR | 128 | 0 | file library
FILTYPE | CHAR | 4 | 0 | file type
FILNAME | CHAR | 128 | 0 | file name
FILVER | SMALLINT | 2 | 0 | file version
RECFMT | CHAR | 10 | 0 | format of record
FLDRECID | INTEGER | 4 | 0 | field ID
FLDDISPL | INTEGER | 4 | 0 | field displ.
FLDNAME | CHAR | 128 | 0 | field name
FLDLEN | INTEGER | 4 | 0 | field length
FLDINT | SMALLINT | 2 | 0 | field integer
FLDDEC | SMALLINT | 2 | 0 | field decimal
FLDSIGN | CHAR | 1 | 0 | field sign
FLDTYPE | CHAR | 1 | 0 | field type
FLDNAT | CHAR | 1 | 0 | field nature
FLJUSTIFY | CHAR | 1 | 0 | justify flag
CLATYPE | SMALLINT | 2 | 0 | field class
INFOPROV | CHAR | 3 | 0 | information origin
MINVAL | CHAR | 33 | 0 | minimum value
MAXVAL | CHAR | 33 | 0 | maximum value
AVGVAL | DECIMAL | 18 | 0 | average value
SPEDIFVAL | INTEGER | 4 | 0 | number of different values
USERPGM | CHAR | 10 | 0 | user program
HASSPE | CHAR | 1 | 0 | spectrum presence flag
FLSPE | CHAR | 1 | 0 | spectrum calculation flag
LOGMON | CHAR | 1 | 0 | logging enabled
DCMON | CHAR | 1 | 0 | data corruption enabled
LOGALARM | CHAR | 1 | 0 | logging alarm
DCALARM | CHAR | 1 | 0 | data corruption alarm
NULLCAP | CHAR | 1 | 0 | null capable field
VARLEN | CHAR | 1 | 0 | field with variable length
ISANAG | CHAR | 1 | 0 | demographic field
ESTCARD | INTEGER | 4 | 0 | estimated cardinality
ESTCLA | SMALLINT | 2 | 0 | estimated class
SAMPVAL | CHAR | 33 | 0 | sample value
SAMPNBR | INTEGER | 4 | 0 | sample OCCURS
FLDTXT | CHAR | 50 | 0 | text
ISKEY | CHAR | 1 | 0 | primary key
FLCLAVAL | CHAR | 1 | 0 | use class value
ANAGPRO | CHAR | 3 | 0 | demographic provider
DATETIME | TIMESTMP | 10 | 0 | logging date/time
Parent topic: Table and Column Descriptions


ANURDCLA

The ANURDCLA table is used for field/class relationships and is populated during the Class Field Assignment phase. The following table describes the columns in the ANURDCLA table:

Column | Type | Length | Decimal | Domain
MCRECID | BINARY | 4 | 0 | machine/company ID
FILRECID | BINARY | 9 | 0 | file ID
FLDRECID | BINARY | 9 | 0 | field ID
CLATYPE | BINARY | 3 | 0 | class identifier
PRG | BINARY | 3 | 0 | progressive number
CDRULECLA | BINARY | 5 | 0 | code rule class assignment
CONFID | BINARY | 9 | 0 | confidence
FLAGDEL | CHARACTER | 1 | 0 | flag deleted
BESTCLA | CHARACTER | 1 | 0 | best class
Parent topic: Table and Column Descriptions


ANURDFLC

The ANURDFLC table is used for copybook/include fields and is populated during the Analyze Copybook phase. The following table describes the columns in the ANURDFLC table:

Column | Type | Length | Decimal | Domain
MCRECID | BINARY | 4 | 0 | machine/company ID
CPYLIB | CHARACTER | 44 | 0 | copybook library
CPYNAME | CHARACTER | 10 | 0 | copybook name
STMTNBR | BINARY | 7 | 0 | statement number
IOAREANAME | CHARACTER | 30 | 0 | I/O area name
IOAREAPRG | BINARY | 3 | 0 | I/O area progressive
IOAREASCP | BINARY | 3 | 0 | I/O area scope
FLDDISPL | BINARY | 5 | 0 | field displacement
FLDNAME | CHARACTER | 128 | 0 | field name
FLDSCP | BINARY | 3 | 0 | field scope
FLDLEVEL | CHARACTER | 2 | 0 | field level
IOAREANBR | BINARY | 5 | 0 | I/O area number
FLDNBR | BINARY | 5 | 0 | field number
FLDLEN | BINARY | 5 | 0 | field length
FLDINT | BINARY | 3 | 0 | field integer
FLDDEC | BINARY | 3 | 0 | field decimal
FLDSIGN | CHARACTER | 1 | 0 | field sign
FLDTYPE | CHARACTER | 1 | 0 | field type
FLDOCC | BINARY | 5 | 0 | field OCCURS
FLDOCCTOT | BINARY | 5 | 0 | total OCCURS elements
FLDOCCLVL | BINARY | 3 | 0 | OCCURS level
FLDOCCPRG | BINARY | 5 | 0 | OCCURS progressive
FLDOCCSHF | BINARY | 5 | 0 | shift to next element displacement
FLDREDEF | CHARACTER | 1 | 0 | field REDEFINES
FLDREDEFC | CHARACTER | 1 | 0 | field REDEFINES clause
FLDNBRREDE | BINARY | 5 | 0 | number of REDEFINES field
FLDGROUP | CHARACTER | 1 | 0 | group field
FLJUSTIFY | CHARACTER | 1 | 0 | justify flag
Parent topic: Table and Column Descriptions


ANURDFLF

The ANURDFLF table is used for file fields and is populated during the Load DB2 Table Information and Analyze File/Copybook Relationship phases. The following table describes the columns in the ANURDFLF table:

Column | Type | Length | Decimal | Domain
MCRECID | SMALLINT | 2 | 0 | machine/company ID
FILRECID | INTEGER | 4 | 0 | file ID
FLDRECID | INTEGER | 4 | 0 | field ID
FLDDISPL | INTEGER | 4 | 0 | field displ.
FLDNAME | CHAR | 128 | 0 | field name
FLDLEN | INTEGER | 4 | 0 | field length
FLDINT | SMALLINT | 2 | 0 | field integer
FLDDEC | SMALLINT | 2 | 0 | field decimal
FLDSIGN | CHAR | 1 | 0 | field sign
FLDTYPE | CHAR | 1 | 0 | field type
FLDNAT | CHAR | 1 | 0 | field nature
FLJUSTIFY | CHAR | 1 | 0 | Justify flag
CLATYPE | SMALLINT | 2 | 0 | field class
INFOPROV | CHAR | 3 | 0 | information origin
MINVAL | CHAR | 33 | 0 | minimum value
MAXVAL | CHAR | 33 | 0 | maximum value
AVGVAL | DECIMAL | 18 | 0 | average value
SPEDIFVAL | INTEGER | 4 | 0 | number of different values
USERPGM | CHAR | 10 | 0 | user program
HASSPE | CHAR | 1 | 0 | spectrum presence flag
FLSPE | CHAR | 1 | 0 | spectrum calculation flag
LOGMON | CHAR | 1 | 0 | logging enabled
DCMON | CHAR | 1 | 0 | data corruption enabled
LOGALARM | CHAR | 1 | 0 | logging alarm
DCALARM | CHAR | 1 | 0 | data corruption alarm
NULLCAP | CHAR | 1 | 0 | null capable field
VARLEN | CHAR | 1 | 0 | field with variable length
ISANAG | CHAR | 1 | 0 | demographic field
ESTCARD | INTEGER | 4 | 0 | estimated cardinality
ESTCLA | SMALLINT | 2 | 0 | estimated class
SAMPVAL | CHAR | 33 | 0 | sample value
SAMPNBR | INTEGER | 4 | 0 | sample OCCURS
FLDTXT | CHAR | 50 | 0 | text
ISKEY | CHAR | 1 | 0 | primary key
FLCLAVAL | CHAR | 1 | 0 | use class value
ANAGPRO | CHAR | 3 | 0 | demographic provider
TYPNAME | CHAR | 128 | 0 | ODBC Data Type Name
TYPNUM | DECIMAL | 5 | 0 | ODBC Data Type Number
Parent topic: Table and Column Descriptions


ANURDFLN

The ANURDFLN table is used for Oracle nested tables and is populated during the Distributed loader phase. The following table describes the columns in the ANURDFLN table:
Column | Type | Length | Decimal | Domain
MCRECID | SMALLINT | 2 | 0 | machine/company ID
FILRECID | INTEGER | 4 | 0 | file ID
FLDRECID | INTEGER | 4 | 0 | field ID
FLDRECIDN | INTEGER | 4 | 0 | field ID for nested
FLDNAME | CHAR | 128 | 0 | field name
TYPNAME | CHAR | 128 | 0 | ODBC Data Type Name
FLDLEN | INTEGER | 4 | 0 | field length
FLDINT | SMALLINT | 2 | 0 | field integer
FLDDEC | SMALLINT | 2 | 0 | field decimal
FLDSIGN | CHAR | 1 | 0 | field sign
FLDTYPE | CHAR | 1 | 0 | field type
FLDNAT | CHAR | 1 | 0 | field nature
FLJUSTIFY | CHAR | 1 | 0 | Justify flag
NULLCAP | CHAR | 1 | 0 | Null capable field
Parent topic: Table and Column Descriptions


ANURDFXC

The ANURDFXC table is used for copybook fields, and is populated during the Life Cycle phase. The following table describes the columns in the ANURDFXC table:

Column | Type | Length | Decimal | Domain
MCRECID | CHARACTER | 4 | 0 | machine/company ID
CPYLIB | BINARY | 44 | 0 | copybook library
CPYNAME | CHARACTER | 10 | 0 | copybook name
STMTNBR | BINARY | 7 | 0 | statement number
IOAREANAME | BINARY | 30 | 0 | I/O area name
IOAREAPRG | BINARY | 3 | 0 | I/O area progressive
IOAREASCP | BINARY | 3 | 0 | I/O area scope
FLDDISPL | BINARY | 5 | 0 | field displacement
FLDNAME | CHARACTER | 128 | 0 | field name
FLDSCP | CHARACTER | 3 | 0 | field scope
FLDLEVEL | BINARY | 2 | 0 | field level
IOAREANBR | BINARY | 5 | 0 | I/O area number
FLDNBR | BINARY | 5 | 0 | field number
FLDLEN | BINARY | 5 | 0 | field length
FLDINT | BINARY | 3 | 0 | field integer
FLDDEC | CHARACTER | 3 | 0 | field decimal
FLDSIGN | CHARACTER | 1 | 0 | field sign
FLDTYPE | CHARACTER | 1 | 0 | field type
FLDOCC | BINARY | 5 | 0 | field OCCURS
FLDOCCTOT | BINARY | 5 | 0 | total OCCURS elements
FLDOCCLVL | BINARY | 3 | 0 | OCCURS level
FLDOCCPRG | BINARY | 5 | 0 | OCCURS progressive
FLDOCCSHF | BINARY | 5 | 0 | shift to next element displacement
FLDREDEF | CHARACTER | 1 | 0 | field REDEFINES
FLDREDEFC | CHARACTER | 1 | 0 | field REDEFINES clause
FLDNBRREDE | BINARY | 5 | 0 | number of REDEFINES field
FLDGROUP | CHARACTER | 1 | 0 | group field
FLJUSTIFY | CHARACTER | 1 | 0 | justify flag
Parent topic: Table and Column Descriptions


ANURDFXF

The ANURDFXF table is used for file fields, and is populated during the Life Cycle phase. The following table describes the columns in the ANURDFXF table:

Column | Type | Length | Decimal | Domain
MCRECID | SMALLINT | 2 | 0 | machine/company ID
FILRECID | INTEGER | 4 | 0 | file ID
FLDRECID | INTEGER | 4 | 0 | field ID
FLDDISPL | INTEGER | 4 | 0 | field displ.
FLDNAME | CHAR | 128 | 0 | field name
FLDLEN | INTEGER | 4 | 0 | field length
FLDINT | SMALLINT | 2 | 0 | field integer
FLDDEC | SMALLINT | 2 | 0 | field decimal
FLDSIGN | CHAR | 1 | 0 | field sign
FLDTYPE | CHAR | 1 | 0 | field type
FLDNAT | CHAR | 1 | 0 | field nature
FLJUSTIFY | CHAR | 1 | 0 | justify flag
CLATYPE | SMALLINT | 2 | 0 | field class
INFOPROV | CHAR | 3 | 0 | information origin
MINVAL | CHAR | 33 | 0 | minimum value
MAXVAL | CHAR | 33 | 0 | maximum value
AVGVAL | DECIMAL | 18 | 0 | average value
SPEDIFVAL | INTEGER | 4 | 0 | number of different values
USERPGM | CHAR | 10 | 0 | user program
HASSPE | CHAR | 1 | 0 | spectrum presence flag
FLSPE | CHAR | 1 | 0 | spectrum calcululation flag
LOGMON | CHAR | 1 | 0 | logging enabled
DCMON | CHAR | 1 | 0 | data corruption enabled
LOGALARM | CHAR | 1 | 0 | logging alarm
DCALARM | CHAR | 1 | 0 | data corruption alarm
NULLCAP | CHAR | 1 | 0 | null capable field
VARLEN | CHAR | 1 | 0 | field with variable length
ISANAG | CHAR | 1 | 0 | demographic field
ESTCARD | INTEGER | 4 | 0 | estimated cardinality
ESTCLA | SMALLINT | 2 | 0 | estimated class
SAMPVAL | CHAR | 33 | 0 | sample value
SAMPNBR | INTEGER | 4 | 0 | sample OCCURS
FLDTXT | CHAR | 50 | 0 | text
ISKEY | CHAR | 1 | 0 | primary key
FLCLAVAL | CHAR | 1 | 0 | use class value
ANAGPRO | CHAR | 3 | 0 | demographic provider
TYPNAME | CHAR | 128 | 0 | ODBC Data Type Name
TYPNUM | DECIMAL | 5 | 0 | ODBC Data Type Number
Parent topic: Table and Column Descriptions


ANURDIMC

The ANURDIMC table is used for copybook/include images, and is populated during the Analyze Copybook phase. The following table describes the columns in the ANURDIMC table:

Column | Type | Length | Decimal | Domain
MCRECID | BINARY | 4 | 0 | machine/company ID
CPYLIB | CHARACTER | 44 | 0 | copybook PDS
CPYNAME | CHARACTER | 10 | 0 | copybook name
IOAREANAME | CHARACTER | 30 | 0 | I/O area name
IOAREAPRG | BINARY | 3 | 0 | I/O area progressive
IMAGEPRG | BINARY | 4 | 0 | image number
RECLEN | BINARY | 5 | 0 | length
IMAGE | CHARACTER | 3900 | 0 | image
Parent topic: Table and Column Descriptions


ANURDIMG

The ANURDIMG table is used for file images and is populated during the Analyze File phase. The following table describes the columns in the ANURDIMG table:

Column | Type | Length | Decimal | Domain
MCRECID | BINARY | 4 | 0 | machine/company ID
FILRECID | BINARY | 9 | 0 | file ID
IMAGEPRG | BINARY | 4 | 0 | image number
IMAGE | CHARACTER | 4000 | 0 | image
Parent topic: Table and Column Descriptions


ANURDIXC

The ANURDIXC table is used for copybook images and is populated during the Life Cycle phase. The following table describes the columns in the ANURDIXC table:

Column | Type | Length | Decimal | Domain
MCRECID | BINARY | 4 | 0 | machine/company ID
CPYLIB | CHARACTER | 44 | 0 | copybook PDS
CPYNAME | CHARACTER | 10 | 0 | copybook name
IOAREANAME | CHARACTER | 30 | 0 | I/O area name
IOAREAPRG | BINARY | 3 | 0 | I/O area progressive
IMAGEPRG | BINARY | 4 | 0 | image number
RECLEN | BINARY | 5 | 0 | length
IMAGE | CHARACTER | 3900 | 0 | image
Parent topic: Table and Column Descriptions


ANURDLOG

The ANURDLOG table is used for log messages and is populated during the Life Cycle phase. The following table describes the columns in the ANURDLOG table:

Column | Type | Length | Decimal | Domain
DTCALC | CHAR | 8 | 0 | last recalculation date
MACHINEID | CHAR | 10 | 0 | machine ID
CDSOC | CHAR | 10 | 0 | company
MCRECID | SMALLINT | 2 | 0 | machine/company ID
FILRECID | INTEGER | 4 | 0 | file ID
FLDNAME | CHAR | 128 | 0 | field name
FLDDISPL | INTEGER | 4 | 0 | field position
FLDVAL | CHAR | 33 | 0 | field value
DATEFMT | CHAR | 10 | 0 | date format
MSGMNE | CHAR | 5 | 0 | message type
MSGID | CHAR | 7 | 0 | message code
MSGDESC | CHAR | 132 | 0 | message description
OCCURS | CHAR | 9 | 0 | OCCURS
GRAVITY | CHAR | 2 | 0 | severity
FLSTOP | CHAR | 1 | 0 | stop elaboration flag
TRCIMG | VARCHAR | 512 | 0 | record image
TRCRRN | INTEGER | 4 | 0 | internal RRN
IMPDTTM | TIMESTMP | 10 | 0 | logging date/time
Parent topic: Table and Column Descriptions


ANURDRFC

The ANURDRFC table is used for file/copybook relationships and is populated during the Analyze File/Copybook Relationship phase. The following table describes the columns in the ANURDRFC table:

Column | Type | Length | Decimal | Domain
MCRECID | BINARY | 4 | 0 | machine/company ID
FILRECID | BINARY | 9 | 0 | file ID
CPYLIB | CHARACTER | 44 | 0 | copybook PDS
CPYNAME | CHARACTER | 10 | 0 | copybook name
IOAREANAME | CHARACTER | 30 | 0 | I/O area name
IOAREAPRG | BINARY | 3 | 0 | I/O area progressive
DELETED | CHARACTER | 1 | 0 | deleted flag
RECLEN | BINARY | 5 | 0 | record length
RECMATCH | BINARY | 5 | 0 | number bytes matching
INFOPROV | CHARACTER | 3 | 0 | information provider
ASSOCIATED | CHARACTER | 1 | 0 | copybook associated
CPYMCRECID | BINARY | 4 | 0 | copybook machine/ company ID
Parent topic: Table and Column Descriptions


ANURDSNC

The ANURDSNC table is used for sampling example field classes and is populated during the Assign Field Class phase. The following table describes the columns in the ANURDSNC table:

Column | Type | Length | Decimal | Domain
CLATYPE | BINARY | 3 | 0 | class identifier
PRG | BINARY | 3 | 0 | progressive number
DTCALC | CHARACTER | 8 | 0 | recalculation date
MCRECID | BINARY | 4 | 0 | machine/company ID
FILRECID | BINARY | 9 | 0 | file ID
FLDRECID | BINARY | 9 | 0 | field ID
FLDLEN | BINARY | 5 | 0 | field length
FLDDGT | BINARY | 3 | 0 | field digit
FLDINT | BINARY | 3 | 0 | field integer
FLDDEC | BINARY | 3 | 0 | field fecimal
FLDTYPE | CHARACTER | 1 | 0 | field type
FLDNAT | CHARACTER | 1 | 0 | field nature
DETAILLVL | CHARACTER | 1 | 0 | detail level
ISANAG | CHARACTER | 1 | 0 | flag demographic field
ALLCOMP | CHARACTER | 1 | 0 | flag sample valid for all companies
IMAGEPRG | BINARY | 4 | 0 | image number
IMAGE | CHARACTER | 721 | 0 | image
Parent topic: Table and Column Descriptions


ANURDSPE

The ANURDSPE table is used for file sampling and is populated during the File Sampling phase. The following table describes the columns in the ANURDSPE table:

Column | Type | Length | Decimal | Domain
DTCALC | CHAR | 8 | 0 | recalculation date
MCRECID | SMALLINT | 2 | 0 | machine/company ID
FILRECID | INTEGER | 4 | 0 | file ID
FLDRECID | INTEGER | 4 | 0 | field ID
SPEVAL | CHAR | 33 | 0 | field value
VALNBR | INTEGER | 4 | 0 | occurence number
DELTANBR | INTEGER | 4 | 0 | in range occurrencies
Parent topic: Table and Column Descriptions


ANURDSPN

The ANURDSPN table is used for compressed file sampling and is populated during the File Sampling phase. The following table describes the columns in the ANURDSPN table:

Column | Type | Length | Decimal | Domain
DTCALC | CHARACTER | 8 | 0 | last recalculation date
MCRECID | BINARY | 4 | 0 | machine/company ID
FILRECID | BINARY | 9 | 0 | file ID
FLDRECID | BINARY | 9 | 0 | field ID
DETAILLVL | CHARACTER | 1 | 0 | detail level
IMAGEPRG | BINARY | 4 | 0 | image number
IMAGE | CHARACTER | 720 | 0 | image
Parent topic: Table and Column Descriptions


ANURDSTR

The ANURDSTR table is used for file record numbers and is populated during the File Sampling phase. The following table describes the columns in the ANURDSTR table:

Column | Type | Length | Decimal | Domain
DTCALC | CHARACTER | 8 | 0 | last recalculation date
MCRECID | BINARY | 4 | 0 | machine/company ID
FILRECID | BINARY | 9 | 0 | file ID
RECNUM | BINARY | 9 | 0 | number of processed records
Parent topic: Table and Column Descriptions


DDDTMCOL

The DDDTMCOL table is used for the Data Dictionary - COL. field and is for internal use only. The following table describes the columns in the DDDTMCOL table:

Column | Type | Length | Decimal | Domain
COLID | DECIMAL | 18 | 0 | field ID
COLSTQID | DECIMAL | 18 | 0 | referenced query ID
COLNAME | CHAR | 18 | 0 | field name
COLSDESC | CHAR | 15 | 0 | short description
COLLDESC | CHAR | 30 | 0 | long description
COLVISIBLE | CHAR | 1 | 0 | field visible
COLCLASS | CHAR | 10 | 0 | field class
COLTAGID | DECIMAL | 18 | 0 | tag ID
Parent topic: Table and Column Descriptions


DDDTMDEC

The DDDTMDEC table is used for the Data Dictionary - field Decode and is for internal use only. The following table describes the columns in the DDDTMDEC table:

Column | Type | Length | Decimal | Domain
DEC_ID | DECIMAL | 18 | 0 | field decode ID
DEC_REFID | DECIMAL | 18 | 0 | field ID
DEC_VALUE | CHAR | 10 | 0 | field decode value
DEC_DISPL | CHAR | 30 | 0 | field decode description
DEC_ICON | DECIMAL | 18 | 0 | icon for decode value
Parent topic: Table and Column Descriptions


DDDTMREF

The DDDTMREF table is used for the Data Dictionary - Ref. field and is for internal use only. The following table describes the columns in the DDDTMREF table:

Column | Type | Length | Decimal | Domain
REF_ID | DECIMAL | 18 | 0 | field ID
REF_STQID | DECIMAL | 18 | 0 | referenced query ID
REF_NAME | CHAR | 18 | 0 | field name
REF_SDESC | CHAR | 15 | 0 | short description
REF_LDESC | CHAR | 30 | 0 | long description
REF_ISTRA | CHAR | 1 | 0 | is transcoded
REF_TAGID | DECIMAL | 18 | 0 | tag ID
Parent topic: Table and Column Descriptions


DDDTMSTQ

The DDDTMSTQ table is used for Data Dictionary - Query information and is for internal use only. The following table describes the columns in the DDDTMSTQ table:

Column | Type | Length | Decimal | Domain
STQ_ID | DECIMAL | 18 | 0 | query ID
STQ_NAME | CHAR | 44 | 0 | table name
STQ_KEYFLD | CHAR | 30 | 0 | key field name
STQ_VIEWFLD | CHAR | 30 | 0 | description field name
STQ_PARFLD | CHAR | 30 | 0 | self reference field name
STQ_TAGID | DECIMAL | 18 | 0 | reference attributes
Parent topic: Table and Column Descriptions


DDDTMTAG

The DDDTMTAG table is used for the Data Dictionary - Field Tag attribute and is for internal use only. The following table describes the columns in the DDDTMTAG table:

Column | Type | Length | Decimal | Domain
TAG_ID | DECIMAL | 18 | 0 | tag ID
TAG_ATTRIB | VARCHAR | 4000 | 0 | tag attributes
Parent topic: Table and Column Descriptions


HSDCHCLA

The HSDCHCLA table is used for the class list extension and is populated during the Work with Class phase. The following table describes the columns in the HSDCHCLA table:

Column | Type | Length | Decimal | Domain
CLATYPE | BINARY | 3 | 0 | class identifier
CHGPGM | CHARACTER | 8 | 0 | Data Masking routine
METHOD | CHARACTER | 10 | 0 | method
MCRECID | BINARY | 4 | 0 | machine/company ID
Parent topic: Table and Column Descriptions


HSDCHFIL

The HSDCHFIL table is used for the file list extension and is populated during the Work with File phase. The following table describes the columns in the HSDCHFIL table:

Column | Type | Length | Decimal | Domain
MCRECID | BINARY | 4 | 0 | machine/company ID
FILRECID | BINARY | 9 | 0 | file ID
PROCIDINP | CHARACTER | 5 | 0 | process ID for input file
UNLINPTYP | CHARACTER | 4 | 0 | unload input file type
UNLINPNAM | CHARACTER | 44 | 0 | unload input file name
UNLINPVER | BINARY | 3 | 0 | unload input file version
FMTSEL | CHARACTER | 8 | 0 | format selector
SLTPGM | CHARACTER | 8 | 0 | select program
UNLOUTTYP | CHARACTER | 4 | 0 | unload output file type
UNLOUTNAM | CHARACTER | 44 | 0 | unload output file name
UNLOUTVER | BINARY | 3 | 0 | unload output file version
WRTPGM | CHARACTER | 8 | 0 | write program for output
METHOD | CHARACTER | 10 | 0 | method
Parent topic: Table and Column Descriptions


HSDGNCOD

The HSDGNCOD table is used to store Data Generator codes. The following table describes the columns in the HSDGNCOD table:

Column | Type | Length | Decimal | Domain
CODE | CHARACTER | 20 | 0 | Code name
MCRECID | BINARY | 9 | 0 | Machine/Company ID
CODETXT | CHARACTER | 50 | 0 | Code Description
CREATOR | CHARACTER | 10 | 0 | Code Creator
Parent topic: Table and Column Descriptions


HSDGNDIR

The HSDGNDIR table is used to store Data Generator directives. The following table describes the columns in the HSDGNDIR table:

Column | Type | Length | Decimal | Domain
CODE | CHARACTER | 20 | 0 | Code name
MCRECID | BINARY | 9 | 0 | Machine/Company ID
FILRECID | BINARY | 9 | 0 | File ID
INCLATYPE | BINARY | 9 | 0 | Input Class
INPRG | BINARY | 9 | 0 | Input Progressive
INFLDRECID | BINARY | 9 | 0 | Input Field ID
OUTCLATYPE | BINARY | 9 | 0 | Output Class
OUTPRG | BINARY | 9 | 0 | Output Progressive
OUTFLDRECID | BINARY | 9 | 0 | Output Field ID
RECCOUNT | BINARY | 9 | 0 | Record Count
FLDNAME | CHARACTER | 128 | 0 | Field Name
DIRTYPE | BINARY | 4 | 0 | Directive Type
ROUTNAME | CHARACTER | 20 | 0 | Routine Name
DICTIONARY | CHARACTER | 20 | 0 | Dictionary Name
OFFSET | BINARY | 9 | 0 | Offset
IN_FLDLEN | BINARY | 9 | 0 | Field Length
VALTYPE | BINARY | 4 | 0 | Value Type
VALLENGTH | BINARY | 9 | 0 | Machine/Company ID
VALNUM | BINARY | 9 | 0 | Number of ValuesD
UNIQUENESS | CHARACTER | 1 | 0 | Unique Value
CURRDATE | CHARACTER | 1 | 0 | Current Date
Parent topic: Table and Column Descriptions


HSDGNDIZ

The HSDGNDIZ table is used to store Data Generator dictionary entries. The following table describes the columns in the HSDGNDIZ table:

Column | Type | Length | Decimal | Domain
DIZNAME | CHARACTER | 8 | 0 | Dictionary Name
DIZPRG | BINARY | 4 | 0 | Progressive
DIZVAL | CHARACTER | 254 | 0 | Value
Parent topic: Table and Column Descriptions


HSDGNFIL

The HSDGNFIL table is used to store Data Generator data store information. The following table describes the columns in the HSDGNFIL table:

Column | Type | Length | Decimal | Domain
MCRECID | BINARY | 4 | 0 | Machine/Company ID
FILRECID | BINARY | 9 | 0 | File ID
PROCIDINP | CHARACTER | 5 | 0 | Process ID for Input File
UNLINPTYP | CHARACTER | 4 | 0 | Unload Input File Type
UNLINPNAM | CHARACTER | 44 | 0 | Unload Input File Name
UNLINPVER | BINARY | 4 | 0 | Unload Input File Version
FMTSEL | CHARACTER | 8 | 0 | Format Selector
SLTPGM | CHARACTER | 8 | 0 | Select Program
UNLOUTTYP | CHARACTER | 4 | 0 | Unload Output File Type
UNLOUTNAM | CHARACTER | 44 | 0 | Unload Output File Name
UNLOUTVER | BINARY | 4 | 0 | Unload Output File Version
WRTPGM | CHARACTER | 8 | 0 | Write Program For Output
CODE | CHARACTER | 10 | 0 | Code
Parent topic: Table and Column Descriptions


HSDGNFLT

The HSDGNFLT table is used to store Data Generator directive information. The following table describes the columns in the HSDGNFLT table:

Column | Type | Length | Decimal | Domain
CODE | CHARACTER | 20 | 0 | Code Name
MCRECID | BINARY | 9 | 0 | Machine/Company ID
FILRECID | BINARY | 9 | 0 | File ID
FLDRECID | BINARY | 9 | 0 | Field ID
PRGOCC | BINARY | 9 | 0 | Progressive
OCCVAL | CHARACTER | 254 | 0 | Value
OCCPER | BINARY | 4 | 0 | Percentage
Parent topic: Table and Column Descriptions


HSDGNLAW

The HSDGNLAW table is used to store Data Generator laws. The following table describes the columns in the HSDGNLAW table:

Column | Type | Length | Decimal | Domain
CODE | CHARACTER | 20 | 0 | Code name
MCRECID | BINARY | 9 | 0 | Machine/Company ID
FILRECID | BINARY | 9 | 0 | File ID
FILENAM | CHARACTER | 128 | 0 | File Name
STEP | BINARY | 9 | 0 | Step
OUTREC | CHARACTER | 1 | 0 | Output Record
Parent topic: Table and Column Descriptions


HSENVELB

The HSENVELB table is used for test environment elaboration and is populated during the Work with Method phase. The following table describes the columns in the HSENVELB table:

Column | Type | Length | Decimal | Domain
METHOD | CHARACTER | 10 | 0 | method name
STEP | BINARY | 4 | 0 | step
MCRECID | BINARY | 4 | 0 | machine/company ID
FILRECID | BINARY | 9 | 0 | file ID
OUTCLATYPE | BINARY | 4 | 0 | output class
OUTPRG | BINARY | 4 | 0 | output prog
OUTFLDRECID | BINARY | 9 | 0 | ouput field
OUTREC | CHARACTER | 1 | 0 | output rec
INCLATYPE | BINARY | 4 | 0 | input class
INPRG | BINARY | 4 | 0 | input prog
INFLDRECID | BINARY | 9 | 0 | input field
FLELAB | CHARACTER | 1 | 0 | flag elab
WRTREC | CBINARY | 9 | 0 | written records
Parent topic: Table and Column Descriptions


HSENVEXT

The HSENVEXT table is used for adding test environment information and is populated during the Environment Extraction phase. The following table describes the columns in the HSENVEXT table:

Column | Type | Length | Decimal | Domain
MCRECID | BINARY | 4 | 0 | machine/company ID
FILRECID | BINARY | 9 | 0 | file ID
METHOD | CHARACTER | 10 | 0 | method name
NUMREC | BINARY | 9 | 0 | number of read records
REDBYT | DECIMAL | 18 | 0 | size reduction in bytes (difference in byte between original and reduced file size)
REDPERC | DECIMAL | 5 | 2 | size reduction percentage
CPUTIME | DECIMAL | 11 | 6 | CPU time in seconds
ELPTIME | DECIMAL | 11 | 6 | elapsed time in seconds
DATESIM | CHAR | 8 | 0 | date of last simulated extraction
DATEEXT | CHAR | 8 | 0 | date of last real extraction
Parent topic: Table and Column Descriptions


HSENVFLT

The HSENVFLT table is used for test environment filtering and is populated during the Work with Method phase. The following table describes the columns in the HSENVFLT table:

Column | Type | Length | Decimal | Domain
METHOD | CHARACTER | 10 | 0 | method name
MCRECID | BINARY | 4 | 0 | machine/company ID
CLATYPE | BINARY | 4 | 0 | class identifier
FLTPRG | BINARY | 4 | 0 | filter prog
FLTTYPE | CHARACTER | 1 | 0 | filter type
MACNBR | BINARY | 4 | 0 | macro number
ROUTNAME | CHARACTER | 8 | 0 | routine name
FLTTXT | CHARACTER | 50 | 0 | filter description
FLTDOC | CHARACTER | 254 | 0 | link to documentation file related to filter
Parent topic: Table and Column Descriptions


HSENVGRP

The HSENVGRP table is used for the test environment method and is populated during the Work with Method phase. The following table describes the columns in the HSENVGRP table:

Column | Type | Length | Decimal | Domain
GROUPNAME | CHARACTER | 10 | 0 | group name
MCRECID | BINARY | 4 | 0 | machine/company ID
GROUPTXT | CHARACTER | 50 | 0 | group description
Parent topic: Table and Column Descriptions


HSENVLMT

The HSENVLMT table is used for storing the cardinality limit of records produced during the subsetting process. It is populated when using Work with Method and Work with Company phases:
Column | Type | Length | Decimal | Domain
METHOD | CHARACTER | 10 | 0 | method name
MCRECID | BINARY | 4 | 0 | machine/company ID
WRNLIMIT | BINARY | 9 | 0 | written record number limit
STRDATE | CHARACTER | 8 | 0 | start date
ENDDATE | CHARACTER | 8 | 0 | end date
Parent topic: Table and Column Descriptions


HSENVMTH

The HSENVMTH table is used for the test environment method and is populated in the Work with Method phase. The following table describes the columns in the HSENVMTH table:

Column | Type | Length | Decimal | Domain
METHOD | CHARACTER | 10 | 0 | method name
MCRECID | BINARY | 4 | 0 | machine/company ID
METHODTXT | CHARACTER | 50 | 0 | method text
DTCALC | CHARACTER | 8 | 0 | last date
GROUPNAME | CHARACTER | 10 | 0 | group
CREATOR | CHARACTER | 10 | 0 | creator
ACTIVE | CHARACTER | 1 | 0 | active
COMPLETED | CHARACTER | 1 | 0 | completed
MTHDOC | CHARACTER | 254 | 0 | link to documentation file related to method
Parent topic: Table and Column Descriptions


HSENVPAR

The HSENVPAR table is used for the test environment macro parameter and is populated during the Work with Method phase. The following table describes the columns in the HSENVPAR table:

Column | Type | Length | Decimal | Domain
METHOD | CHARACTER | 10 | 0 | method name
MCRECID | BINARY | 4 | 0 | machine/company ID
CLATYPE | BINARY | 4 | 0 | class identifier
FLTPRG | BINARY | 4 | 0 | filter progressive
PARMPRG | BINARY | 4 | 0 | parameter progressive
VALUE | CHARACTER | 254 | 0 | parameter value
Parent topic: Table and Column Descriptions


HSENVPRN

The HSENVPRN table is used for the test environment parent segment and is populated during the Work with Method phase. The following table describes the columns in the HSENVPRN table:

Column | Type | Length | Decimal | Domain
MCRECID | BINARY | 4 | 0 | machine/company ID
FILRECID | BINARY | 8 | 0 | file ID
Parent topic: Table and Column Descriptions


HSENVRRD

The HSENVRRD table is used for storing reload rules, and is populated during the Work with Method phase. The following table describes the columns in the HSENVRRD table:

Column | Type | Length | Decimal | Domain
METHOD | CHARACTER | 10 | 0 | method name
MCRECID | BINARY | 4 | 0 | machine/company ID
RELRULE | CHARACTER | 10 | 0 | rule name
FILRECID | BINARY | 9 | 0 | file ID
REPL | CHARACTER | 1 | 0 | replace
Parent topic: Table and Column Descriptions


HSENVRRH

The HSENVRRH table is used for storing reload rules, and is populated during the Work with Method phase. The following table describes the columns in the HSENVRRH table:

Column | Type | Length | Decimal | Domain
METHOD | CHARACTER | 10 | 0 | method name
MCRECID | BINARY | 4 | 0 | machine/company ID
RELRULE | CHARACTER | 10 | 0 | rule name
RELRULETXT | CHARACTER | 50 | 0 | rule text
SKLLOAD | CHARACTER | 8 | 0 | skeleton for LOAD
SKLREP | CHARACTER | 8 | 0 | skeleton for REPAIR
SKLREP | CHARACTER | 8 | 0 | skeleton for REPAIR
MAXSTEP | BINARY | 4 | 0 | maximum number of steps per JCL
PDSNAME | CHARACTER | 44 | 0 | PDS name for generated JCLs
MBRLOAD | CHARACTER | 5 | 0 | member for LOAD
MBRREP | CHARACTER | 5 | 0 | member for REPAIR
CRTSYS | CHARACTER | 1 | 0 | create STSPUNCH
CRTLOA | CHARACTER | 1 | 0 | create LOAD
TGTOWNER | CHARACTER | 128 | 0 | target OWNER
TGTDSN | CHARACTER | 5 | 0 | target DSN
Parent topic: Table and Column Descriptions


HSENVSTP

The HSENVSTP table is used for test environment steps and is populated during the Work with Method phase. The following table describes the columns in the HSENVSTP table:

Column | Type | Length | Decimal | Domain
METHOD | CHARACTER | 10 | 0 | method name
STEP | BINARY | 4 | 0 | step name
MCRECID | BINARY | 4 | 0 | machine/company ID
STEPTXT | CHARACTER | 50 | 0 | step text
FLELAB | CHARACTER | 1 | 0 | flag elab
DTCALC | CHARACTER | 8 | 0 | last date
TIMECALC | CHARACTER | 8 | 0 | last time
STPDOC | CHARACTER | 254 | 0 | link to documentation file related to step
Parent topic: Table and Column Descriptions


HSLICCHG

The HSLICCHG table is used for applying changes and is populated during the Life Cycle phase. The following table describes the columns in the HSLICCHG table:

Column | Type | Length | Decimal | Domain
MCRECID | BINARY | 4 | 0 | machine/company ID
BKPMCRECID | BINARY | 4 | 0 | backup machine/company ID
FLACTIVE | CHARACTER | 1 | 0 | flag active
Parent topic: Table and Column Descriptions


HSLICFIL

The HSLICFIL table is used for changed files and is populated during the Life Cycle phase. The following table describes the columns in the HSLICFIL table:

Column | Type | Length | Decimal | Domain
MCRECID | BINARY | 4 | 0 | machine/Company ID
MACHINEID | CHARACTER | 10 | 0 | machine ID
CDSOC | CHARACTER | 10 | 0 | company ID
FILRECID | BINARY | 9 | 0 | file ID
FILLIB | CHARACTER | 128 | 0 | file library
FILTYPE | CHARACTER | 4 | 0 | file type
FILNAME | CHARACTER | 128 | 0 | file name
FILVER | BINARY | 3 | 0 | file version
RECFMT | CHARACTER | 10 | 0 | format of record
APPLID | CHARACTER | 10 | 0 | application ID
PROCID | CHARACTER | 5 | 0 | process ID
UNLFILTYP | CHARACTER | 4 | 0 | unload file type
UNLFILNAM | CHARACTER | 44 | 0 | unload file name
UNLFILVER | BINARY | 3 | 0 | unload file version
FMTSEL | CHARACTER | 8 | 0 | format selected
FILNICK | CHARACTER | 10 | 0 | nickname of file
CPYMCRECID | BINARY | 4 | 0 | copybook machine/company ID
CPYMACHINEID | CHARACTER | 10 | 0 | copybook machine ID
CPYCDSOC | CHARACTER | 10 | 0 | copybook company ID
CPYLIB | CHARACTER | 44 | 0 | copybook library
CPYNAME | CHARACTER | 10 | 0 | copybook name
IOAREANAME | CHARACTER | 30 | 0 | I/O area name
IOAREAPRG | BINARY | 4 | 0 | I/O area prg.
FILSTS | CHARACTER | 1 | 0 | file status
FILCON | CHARACTER | 1 | 0 | file confidence
FILDATE | CHARACTER | 8 | 0 | copybook identification date
FLDSTS | CHARACTER | 1 | 0 | fields status
FLDCON | CHARACTER | 1 | 0 | fields confidence
FLDDATE | CHARACTER | 8 | 0 | fields identification date
APPSTS | CHARACTER | 1 | 0 | apply change status
APPDATE | CHARACTER | 8 | 0 | apply change date
NEWCPY | CHARACTER | 1 | 0 | new copybook
NEWFIL | CHARACTER | 1 | 0 | new file
INFOPROV | CHARACTER | 3 | 0 | information provider
TYPNAME | CHARACTER | 128 | 0 | ODBC Database Type Name
Parent topic: Table and Column Descriptions


HSLICFLD

The HSLICFLD table is used for changed files and is populated during the Life Cycle phase. The following table describes the columns in the HSLICFLD table:

Column | Type | Length | Decimal | Domain
MCRECID | SMALLINT | 2 | 0 | machine/company ID
FILRECID | INTEGER | 4 | 0 | file ID
OLDFLDRECID | INTEGER | 4 | 0 | old field ID
NEWFLDRECID | INTEGER | 4 | 0 | new field ID
FLDCON | CHAR | 1 | 0 | confidence
Parent topic: Table and Column Descriptions


HSLICFTF

The HSLICFTF table is used to store life cycle specifications, and is populated during the life cycle. The following table describes the columns in the HSLICFTF table:

Column | Type | Length | Decimal | Domain
MCRECID1 | BINARY | 4 | 0 | Source Machine/Company ID
FILRECID1 | BINARY | 9 | 0 | Source File ID
MCRECID2 | BINARY | 4 | 0 | Target Machine/Company ID
FILRECID2 | BINARY | 9 | 0 | Target File ID
Parent topic: Table and Column Descriptions


HSLICLID

The HSLICLID table is used for source level IDs and is populated during the Life Cycle phase. The following table describes the columns in the HSLICLID table:

Column | Type | Length | Decimal | Domain
CPYLIB | CHARACTER | 128 | 0 | copybook library
CPYNAME | CHARACTER | 128 | 0 | copybook name
LVLID | CHARACTER | 24 | 0 | level number
NEWLVLID | CHARACTER | 24 | 0 | new level number
DSNAME | CHARACTER | 128 | 0 | Database name
Parent topic: Table and Column Descriptions


HSLICSCD

The HSLICSCD table is currently unused, but is present to accommodate future development. The following table describes the columns in the HSLICSCD table:

Column | Type | Length | Decimal | Domain
PHASE | CHARACTER | 10 | 0 | Phase
SEQUENCE | BINARY | 4 | 0 | Scheduling Sequence
FREQUENCY | CHARACTER | 1 | 0 | Scheduling Frequency
DAYNBR | BINARY | 4 | 0 | Day Number
CPYLIB | CHARACTER | 44 | 0 | Data Set Name
CPYTYPE | CHARACTER | 1 | 0 | Data Set Type
Parent topic: Table and Column Descriptions


HSLICSTR

The HSLICSTR table is used for stratification and is populated during the Life Cycle phase. The following table describes the columns in the HSLICSTR table:

Column | Type | Length | Decimal | Domain
CPYLIB | CHAR TER | 44 | 0 | copybook library
BASCPYLIB | CHAR TER | 44 | 0 | based level
LIBLVL | BINARY | 1 | 0 | Level number
Parent topic: Table and Column Descriptions


HSLOGTAB

The HSLOGTAB table is used for the logging process and is populated by the customer. The following table describes the columns in the HSLOGTAB table:

Column | Type | Length | Decimal | Domain
MSGCODE | CHAR TER | 7 | 0 | Return Code
FLAGENVI | CHAR TER | 7 | 0 | Flag for record writing
ROUTNAME | CHAR TER | 8 | 0 | Logging Routine Name
VALRETCODE | CHAR TER | 2 | 0 | System Variable Value
Parent topic: Table and Column Descriptions


HSSYSAUX

The HSSYSAUX table is used for SYSAUXRELS views. The following table describes the columns in the HSSYSAUX table:

Column | Type | Length | Decimal | Domain
TBOWNER | CHAR | 128 | 0 | owner of the table
TBNAME | CHAR | 128 | 0 | table name
COLNAME | CHAR | 128 | 0 | Column name
PARTITION | BINARY | 4 | 0 | Partition
AUXTBOWNER | CHAR | 128 | 0 | Owner of the auxiliary table
AUXTBNAME | BINARY | 128 | 0 | Auxiliary Table name
AUXRELOBID | BINARY | 4 | 0 | Internal identifier
IBMREQID | CHAR | 1 | 0 |
Parent topic: Table and Column Descriptions


HSSYSCKS

The HSSYSCKS table is used for SYSCHECKS views. The following table describes the columns in the HSSYSCKS table:

Column | Type | Length | Decimal | Domain
TBOWNER | CHAR | 128 | 0 | owner of the table
CREATOR | CHAR | 128 | 0 | Creator name
TBNAME | CHAR | 128 | 0 | Table name
CHECKNAME | CHAR | 128 | 0 | Check constraint name
CHECKCONDITION | CHAR | 3800 | 0 | Text of the check constraint
Parent topic: Table and Column Descriptions


HSSYSCOL

The HSSYSCOL table is used for SYSCOLUMNS views. The following table describes the columns in the HSSYSCOL table:

Column | Type | Length | Decimal | Domain
NAME | CHAR | 128 | 0 | Column Name
TBNAME | CHAR | 128 | 0 | Table Name
TBCREATOR | CHAR | 128 | 0 | Creator name
COLNO | SMALLINT | 4 | 0 | Numeric place of the column
COLTYPE | CHAR | 8 | 0 | Column Type
LENGTH | SMALLINT | 4 | 0 | Column length
SCALE | CHAR | 1 | 0 | Scale of decimal data
NULLS | SMALLINT | 4 | 0 | Whether the column can contain null values
COLCARDS | INTEGER | 4 | 0 |
HIGH2KEY | CHAR | 8 | 0 | Second highest value of the column
LOW2KEY | CHAR | 8 | 0 | Second lowest value of the column
UPDATES | CHAR | 1 | 0 | Whether the column can be updated
IBMREQID | CHAR | 1 | 0 |
REMARKS | VARCHAR | 254 | 0 | A character string provided by the user
DEFAULT | CHAR | 1 | 0 | Default Value
KEYSEQ | SMALLINT | 4 | 0 | The numeric position of the column within the primary key of the table
FOREIGNKEY | CHAR | 1 | 0 | Applies to character or CLOB columns
FLDPROC | CHAR | 1 | 0 | Whether the column has a field procedure
LABEL | VARCHAR | 30 | 0 | The column label provided by the user
STATSTIME | TIMESTAMP | 4 | 0 | RUNSTAT statisctics
DEFAULTVALUE | VARCHAR | 254 | 0 | Default Value
LENGTH2 | SMALLINT | 4 | 0 | Maximum length of the data retrieved from the column
Parent topic: Table and Column Descriptions


HSSYSDAU

The HSSYSDAU table is used for SYSDBAUTH views. The following table describes the columns in the HSSYSDAU table:

Column | Type | Length | Decimal | Domain
GRANTOR | CHAR | 128 | 0 | Authorization ID of the user who granted the privileges
GRANTEE | CHAR | 128 | 0 | Application ID of the user who holds the privilege
NAME | VARCHAR | 24 | 0 | Database name
TIMESTAMP | CHAR | 12 | 0 |
DATEGRANTED | CHAR | 6 | 0 |
TIMEGRANTED | CHAR | 8 | 0 |
GRANTEETYPE | CHAR | 1 | 0 | Indicates the type of grantee
AUTHHOWGOT | CHAR | 1 | 0 | Authorization level of the user from whom the privileges were received
CREATETABAUTH | CHAR | 1 | 0 | Whether the GRANTEE can create tables within the database
CREATETSAUTH | CHAR | 1 | 0 | Whether the GRANTEE can create table spaces within the database
DBADMAUTH | CHAR | 1 | 0 | Whether the GRANTEE has DBADM authority over the database
DBCTRLAUTH | CHAR | 1 | 0 | Whether the GRANTEE has DBCTRL authority over the database
DBMAINTAUTH | CHAR | 1 | 0 | Whether the GRANTEE has DBMAINT authority over the database
DISPLAYDBAUTH | CHAR | 1 | 0 | Whether the GRANTEE can issue the DISPLAY command for the database
DROPAUTH | CHAR | 1 | 0 | Whether the GRANTEE can issue the ALTER DATABASE and DROP DATABASE statement
IMAGCOPYAUTH | CHAR | 1 | 0 | Whether the GRANTEE can use the COPY, MERGECOPY, MODIFY, and QUIESCE utilities on the database
LOADAUTH | CHAR | 1 | 0 | Whether the GRANTEE can use the LOAD utility to load tables in the database
REORGAUTH | CHAR | 1 | 0 | Whether the GRANTEE can use the REORG utility to reorganize table spaces and indexes in the database
RECOVERDBAUTH | CHAR | 1 | 0 | Whether the GRANTEE can use the RECOVER and REPORT utilities on table spaces in the database
REPAIRAUTH | CHAR | 1 | 0 | Whether the GRANTEE can use the DIAGNOSE and REPAIR utilities on table spaces and indexes in the Database
STARTDBAUTH | CHAR | 1 | 0 | Whether the GRANTEE can use the START command against the database
STATSAUTH | CHAR | 1 | 0 | Whether the GRANTEE can use the CHECK and RUNSTATS utilities against the database
STOPAUTH | CHAR | 1 | 0 | Whether the GRANTEE can issue the STOP command against the database
IBMREQD | CHAR | 1 | 0 |
GRANTEDTS | TIMESTAMP | 26 | 0 | Time when the GRANT statement was executed
Parent topic: Table and Column Descriptions


HSSYSDBA

The HSSYSDBA table is used for SYSDATABASE views. The following table describes the columns in the HSSYSDBA table:

Column | Type | Length | Decimal | Domain
NAME | VARCHAR | 24 | 0 | Database name
CREATOR | VARCHAR | 128 | 0 | Authorization ID of the owner of the database
STGROUP | VARCHAR | 128 | 0 | Name of the default storage group of the database
BPOOL | CHAR | 8 | 0 | Name of the default buffer pool of the table space
DBID | SAMLLINT | 4 | 0 | Internal identifier of the database
IBMREQD | CHAR | 1 | 0 |
CREATEDBY | VARCHAR | 128 | 0 | Primary authorization ID of the user who created the database
ROSHARE | CHAR | 1 | 0 |
TIMESTAMP | TIMESTAMP | 26 | 0 |
TYPE | CHAR | 1 | 0 | Type of database
GROUP_MEMBER | VARCHAR | 24 | 0 | The DB2 data sharing member name of the DB2 subsystem that uses this work file database
Parent topic: Table and Column Descriptions


HSSYSDEP

The HSSYSDEP table is used for SYSCHECKDEP views. The following table describes the columns in the HSSYSDEP table:

Column | Type | Length | Decimal | Domain
TBOWNER | VARCHAR | 128 | 0 | The schema of the table on which the check constraint is defined
TBNAME | VARCHAR | 128 | 0 | Name of the table on which the check constraint is defined
CHECKNAME | VARCHAR | 128 | 0 | Name of the check constraint
COLNAME | VARCHAR | 128 | 0 | Name of the column that the check constraint refers to
IBMREQD | CHAR | 1 | 0 |
Parent topic: Table and Column Descriptions


HSSYSFKE

The HSSYSFKE table is used for SYSFOREIGNKEYS views. The following table describes the columns in the HSSYSFKE table:

Column | Type | Length | Decimal | Domain
CREATOR | VARCHAR | 128 | 0 | Schema or qualifier of the table that contains the column
TBNAME | VARCHAR | 128 | 0 | Name of the table on contains the column
RELNAME | VARCHAR | 128 | 0 | Constraint name for the constraint for which the column is part of the foreign key
COLNAME | VARCHAR | 128 | 0 | Name of the column
COLNO | SMALLINT | 4 | 0 | Numeric place of the column in its table
COLSEQ | SMALLINT | 4 | 0 | Numeric place of the column in the foreign key
IBMREQD | CHAR | 1 | 0 |
Parent topic: Table and Column Descriptions


HSSYSIND

The HSSYSIND table is used for SYSINDEXES views. The following table describes the columns in the HSSYSIND table:

Column | Type | Length | Decimal | Domain
NAME | VARCHAR | 128 | 0 | Name of the index
CREATOR | VARCHAR | 128 | 0 | The schema of the index
TBNAME | VARCHAR | 128 | 0 | Name of the table on which the index is defined
TBCREATOR | VARCHAR | 128 | 0 | The schema of the table
UNIQUERULE | CHAR | 1 | 0 | Whether the index is unique
COLCOUNT | SMALLINT | 4 | 0 | The number of columns in the key
CLUSTERING | CHAR | 1 | 0 | Whether CLUSTER was specified when the index was created
CLUSTERED | CHAR | 1 | 0 | Whether the table is actually clustered by the index
DBID | SMALLINT | 4 | 0 | Internal identifier of the database
OBID | SMALLINT | 4 | 0 | Internal identifier of the index fan set descriptor
ISOBID | SMALLINT | 4 | 0 | Internal identifier of the index page set descriptor
DBNAME | VARCHAR | 24 | 0 | Name of the database that contains the index
INDEXSPACE | VARCHAR | 24 | 0 | Name of the index space
FIRSTKEYCARD | INTEGER | 4 | 0 |
FULLKEYCARD | INTEGER | 4 | 0 |
NLEAF | INTEGER | 4 | 0 | Number of active leaf pages in the index
NLEVELS | SMALLINT | 4 | 0 | Number of levels in the index tree
BPOOL | CHAR | 8 | 0 | Name of the buffer pool used for the index
PGSIZE | SMALLINT | 4 | 0 | Contains the value 4, 8, 16, or 32 which indicates the size, in KB, of the leaf pages in the index
ERASERULE | CHAR | 1 | 0 | Whether the data sets are erased when dropped
DSETPASS | CHAR | 8 | 0 |
CLOSERULE | CHAR | 1 | 0 | Whether the data sets are candidates for closure when the limit on the number of open data sets is reached
SPACE | INTEGER | 4 | 0 | Number of kilobytes of DASD storage allocated to the index
IBMREQD | CHAR | 1 | 0 |
CLUSTERRATIO | SMALLINT | 4 | 0 | Percentage of rows that are in clustering order
CREATEDBY | VARCHAR | 128 | 0 | Primary authorization ID of the user who created the index
IOFACTOR | SMALLINT | 4 | 0 |
PREFETCHFACTOR | SMALLINT | 4 | 0 |
STATSTIME | TIMESTAMP | 26 | 0 | RUNSTATS updated the statistics
INDEXTYPE | CHAR | 1 | 0 | index type
Parent topic: Table and Column Descriptions


HSSYSIPA

The HSSYSIPA table is used for SYSINDEXPART views. The following table describes the columns in the HSSYSIPA table:

Column | Type | Length | Decimal | Domain
PARTITION | SMALLINT | 4 | 0 | Partition number
IXNAME | VARCHAR | 128 | 0 | Name of the index
IXCREATOR | VARCHAR | 128 | 0 | The schema of the index
PQTY | INTEGER | 4 | 0 | Primary quantity
SQTY | SMALLINT | 4 | 0 | Secondary quantity
STORTYPE | CHAR | 1 | 0 | Type of storage
STORNAME | VARCHAR | 128 | 0 | Name of storage group
VCATNAME | VARCHAR | 24 | 0 | Name of integrated catalog
CARD | INTEGER | 4 | 0 |
FAROFFPOS | INTEGER | 4 | 0 |
LEAFDIST | INTEGER | 4 | 0 | 100 times the average number of leaf pages between successive active leaf pages of the index
NEAROFFPOS | INTEGER | 4 | 0 |
IBMREQD | CHAR | 1 | 0 |
LIMITKEY | VARCHAR | 512 | 0 | The high value of the limit key of the partition in an internal format
FREEPAGE | SMALLINT | 4 | 0 | Number of pages that are loaded before a page is left as free space
PCTFREE | SMALLINT | 4 | 0 | Percentage of each leaf or nonleaf page that is left as free space
SPACE | INTEGER | 4 | 0 | Number of kilobytes of DASD storage allocated to the index space partition
STATSTIME | TIMESTAMP | 26 | 0 | RUNSTATS updated the statistics
INDEXTYPE | CHAR | 1 | 0 |
GBPCACHE | CHAR | 1 | 0 | Group buffer pool cache option specified for this index or index partition
Parent topic: Table and Column Descriptions


HSSYSKEY

The HSSYSKEY table is used for SYSKEY views. The following table describes the columns in the HSSYSKEY table:

Column | Type | Length | Decimal | Domain
IXNAME | VARCHAR | 128 | 0 | Name of the index
IXCREATOR | VARCHAR | 128 | 0 | Schema or qualifier of the index
COLNAME | VARCHAR | 128 | 0 | Name of the column of the key
COLNO | SMALLINT | 4 | 0 | Numeric position of the column in the table
COLSEQ | SMALLINT | 4 | 0 | Numeric position of the column in the key for an index on columns
ORDERING | CHAR | 1 | 0 | Order of the column in the key
IBMREQD | CHAR | 1 | 0 |
Parent topic: Table and Column Descriptions


HSSYSRAU

The HSSYSRAU table is used for SYSRESAUTH views. The following table describes the columns in the HSSYSRAU table:

Column | Type | Length | Decimal | Domain
GRANTOR | VARCHAR | 128 | 0 | Authorization ID of the user who granted the privilege
GRANTEE | VARCHAR | 128 | 0 | Authorization ID of the user who holds the privilege
QUALIFIER | VARCHAR | 128 | 0 | Qualifier of the table space
NAME | VARCHAR | 128 | 0 | Name of the buffer pool
GRANTEETYPE | CHAR | 1 | 0 |
AUTHHOWGOT | CHAR | 1 | 0 | Authorization level of the user from whom the privileges were received
OBTYPE | CHAR | 1 | 0 | Type of object
TIMESTAMP | CHAR | 12 | 0 |
DATEGRANTED | CHAR | 8 | 0 |
TIMEGRANTED | CHAR | 8 | 0 |
USEAUTH | CHAR | 1 | 0 | Whether the privilege is held with the GRANT option
IBMREQD | CHAR | 1 | 0 |
GRANTEDTS | TIMESTAMP | 26 | 0 | Time when the GRANT statement was executed
Parent topic: Table and Column Descriptions


HSSYSREL

The HSSYSREL table is used for SYSRELS views. The following table describes the columns in the HSSYSREL table:

Column | Type | Length | Decimal | Domain
CREATOR | VARCHAR | 128 | 0 | The schema of the dependent table of the referential constraint
TBNAME | VARCHAR | 128 | 0 | Name of the dependent table of the referential constraint
RELNAME | VARCHAR | 128 | 0 | Constraint name
REFTBNAME | VARCHAR | 128 | 0 | Name of the parent table of the referential constraint
REFTBCREATOR | VARCHAR | 128 | 0 | The schema of the parent table
COLCOUNT | SMALLINT | 4 | 0 | Number of columns in the foreign key
DELETERULE | CHAR | 1 | 0 | Type of delete rule for the referential constraint
IBMREQD | CHAR | 1 | 0 |
RELOBID1 | SMALLINT | 4 | 0 | Internal identifier of the constraint with respect to the database that contains the parent table
RELOBID2 | SMALLINT | 4 | 0 | Internal identifier of the constraint with respect to the database that contains the parent table
TIMESTAMP | TIMESTAMP | 26 | 0 | Date and time the constraint was defined
IXOWNER | VARCHAR | 128 | 0 | The schema of unique non-primary index used for the parent key
IXNAME | VARCHAR | 128 | 0 | Name of unique non-primary index used for a parent key
Parent topic: Table and Column Descriptions


HSSYSSYN

The HSSYSYN table is used for SYSSYNONYMS views. The following table describes the columns in the HSSYSSYN table:

Column | Type | Length | Decimal | Domain
NAME | VARCHAR | 128 | 0 | Synonym for the table or view
CREATOR | VARCHAR | 128 | 0 | Authorization ID of the owner of the synonym
TBNAME | VARCHAR | 128 | 0 | Name of the table or view
TBCREATOR | VARCHAR | 128 | 0 | The schema of the table or view
IBMREQD | CHAR | 1 | 0 |
CREATEDBY | VARCHAR | 128 | 0 | Primary authorization ID of the user who created the synonym
CREATEDTS | TIMESTAMP | 26 | 0 | Time when the CREATE statement was executed for the synonym
Parent topic: Table and Column Descriptions


HSSYSTAB

The HSSYTAB table is used for SYSTABLES views. The following table describes the columns in the HSSYSTAB table:

Column | Type | Length | Decimal | Domain
NAME | VARCHAR | 128 | 0 | Name of the table, view, or alias
CREATOR | VARCHAR | 128 | 0 | The schema of the table, view, or alias
TYPE | CHAR | 1 | 0 | Type of object
DBNAME | VARCHAR | 24 | 0 | For a table, or a view of tables, the name of the database that contains the table space named in TSNAME
TSNAME | VARCHAR | 24 | 0 | For a table, or a view of one table, the name of the table space that contains the table
DBID | SMALLINT | 4 | 0 | Internal identifier of the database
OBID | SMALLINT | 4 | 0 | Internal identifier of the table
COLCOUNT | SMALLINT | 4 | 0 | Number of columns in the table or view
EDPROC | VARCHAR | 24 | 0 | Name of the edit procedure
VALPROC | VARCHAR | 24 | 0 | Name of the validation procedure
CLUSTERTYPE | CHAR | 1 | 0 | Whether RESTRICT ON DROP applies
CLUSTERRID | INTEGER | 4 | 0 |
CARD | INTEGER | 4 | 0 |
NPAGES | INTEGER | 4 | 0 | Total number of pages on which rows of the table appear
PCTPAGES | SMALLINT | 4 | 0 | Percentage of active table space pages that contain rows of the table
IBMREQD | CHAR | 1 | 0 |
REMARKS | VARCHAR | 254 | 0 | A character string provided by the user with the COMMENT statement
PARENTS | SMALLINT | 4 | 0 | Number of relationships in which the table is a dependent
CHILDREN | SMALLINT | 4 | 0 | Number of relationships in which the table is a parent
KEYCOLUMNS | SMALLINT | 4 | 0 | Number of columns in the primary key of the table
RECLENGTH | SMALLINT | 4 | 0 | For user tables, the maximum length of any record in the table
STATUS | CHAR | 1 | 0 | Indicates the status of the table definition
KEYOBID | SMALLINT | 4 | 0 | Internal DB2 identifier of the index that enforces uniqueness of the primary key of the table
LABEL | VARCHAR | 30 | 0 | The label as given by a LABEL statement; otherwise, the value is an empty string
CHECKFLAG | CHAR | 1 | 0 | The table space that contains the table is in a check pending status
CHECKRID | CHAR | 4 | 0 |
AUDITING | CHAR | 1 | 0 | Value of the audit option
CREATEDBY | VARCHAR | 128 | 0 | Primary authorization ID of the user who created the table, view, or alias
LOCATION | VARCHAR | 128 | 0 | Location name of the object of an alias
TBCREATOR | VARCHAR | 128 | 0 | name of the creator
TBNAME | VARCHAR | 128 | 0 | the name of the base table
CREATEDTS | TIMESTAMP | 26 | 0 | Time when the CREATE statement was executed for the table, view, or alias
ALTEREDTS | TIMESTAMP | 26 | 0 | Time when the CREATE statement was executed for the table, view, or alias
DATACAPTURE | CHAR | 1 | 0 | Records the value of the DATA CAPTURE option for a table
RBA1 | CHAR | 6 | 0 | The log RBA when the table was created
RBA2 | CHAR | 6 | 0 | The log RBA when the table was created
PCTROWCOMP | SMALLINT | 4 | 0 | Percentage of rows compressed within the total number of active rows in the table
STATSTIME | TIMESTAMP | 26 | 0 | RUNSTATS updated the statistics
CHECKS | SMALLINT | 4 | 0 | Number of check constraints defined on the table
CARDF | FLOAT | 2 | 0 | Total number of rows in the table or total number of LOBs in an auxiliary table
CHECKRID5B | CHAR | 5 | 0 | Blank if the table or partition is not in a check pending status
ENCODING_SCHEME | CHAR | 1 | 0 | Encoding scheme for tables
TABLESTATUS | VARCHAR | 30 | 0 | Indicates the reason for an incomplete table definition
Parent topic: Table and Column Descriptions


HSSYSTAU

The HSSYTAU table is used for SYSTABAUTH views. The following table describes the columns in the HSSYSTAU table:

Column | Type | Length | Decimal | Domain
GRANTOR | VARCHAR | 128 | 0 | Authorization ID or role of the user who granted the privileges
GRANTEE | VARCHAR | 128 | 0 | Authorization ID of the user who holds the privileges or the name of an application plan or package that uses the privileges
GRANTEETYPE | CHAR | 1 | 0 | Type of grantee
DBNAME | VARCHAR | 24 | 0 | If the privileges were received from a user with DBADM, DBCTRL, or DBMAINT authority, DBNAME is the name of the database on which the GRANTOR has that Authority
SCREATOR | VARCHAR | 128 | 0 | If the row of SYSIBM.SYSTABAUTH was created as a result of a CREATE VIEW statement, SCREATOR is the schema of a table or view referred to in the CREATE VIEW statement
STNAME | VARCHAR | 128 | 0 | the name of a table or view referred to in the fullselect of the CREATE TABLE statement
TCREATOR | VARCHAR | 128 | 0 | The schema of the table or view
TTNAME | VARCHAR1 | 128 | 0 | Name of the table or view
AUTHHOWGOT | CHAR | 1 | 0 | Authorization level of the user from whom the privileges were received
TIMESTAMP | CHAR | 12 | 0 |
DATEGRANTED | CHAR | 6 | 0 |
TIMEGRANTED | CHAR | 8 | 0 |
UPDATECOLS | CHAR | 1 | 0 | The value is blank if the value of UPDATEAUTH applies uniformly to all columns of the table or view. The value is an asterisk (*) if the value of UPDATEAUTH applies to some columns but not to others. In this case, rows will exist in SYSIBM.SYSCOLAUTH with matching timestamps and PRIVILEGE = blank. These rows list the columns on which update privileges have been granted.
ALTERAUTH | CHAR | 1 | 0 | Whether the GRANTEE can alter the table
DELETEAUTH | CHAR | 1 | 0 | Whether the GRANTEE can delete rows from the table or view
INDEXAUTH | CHAR | 1 | 0 | Whether the GRANTEE can create indexes on the table
INSERTAUTH | CHAR | 1 | 0 | Whether the GRANTEE can insert rows into the table or view
SELECTAUTH | CHAR | 1 | 0 | Whether the GRANTEE can select rows from the table or view
UPDATEAUTH | CHAR | 1 | 0 | Whether the GRANTEE can update rows of the table or view
IBMREQD | CHAR | 1 | 0 |
GRANTEELOCATION | CHAR | 16 | 0 |
LOCATION | CHAR | 16 | 0 |
COLLID | CHAR | 18 | 0 | If the GRANTEE is a package, its collection name. Otherwise, the value is blank
CONTOKEN | CHAR | 8 | 0 | If the GRANTEE is a package, the consistency token of the DBRM from which the package was derived. Otherwise, the value is blank
CAPTUREAUTH | CHAR | 1 | 0 |
REFERENCESAUTH | CHAR | 1 | 0 | Whether the GRANTEE can create or drop referential constraints in which the table is a parent
REFCOLS | CHAR | 1 | 0 | The value of this column is blank if the value of REFERENCESAUTH applies uniformly to all columns of the table. The value is an asterisk(*) if the value of REFERENCESAUTH applies to some columns but not to others. In this case, rows will exist in SYSIBM.SYSCOLAUTH with PRIVILEGE = R and matching timestamps that list the columns on which reference privileges have been granted
GRANTEDTS | TIMESTAMP | 26 | 0 | Time when the GRANT statement was executed
TRIGGERAUTH | CHAR | 1 | 0 | Whether the GRANTEE can create triggers in which the table is named as the subject table
Parent topic: Table and Column Descriptions


HSSYSTPA

The HSSYTPA table is used for SYSTABLEPART views. The following table describes the columns in the HSSYSTPA table:

Column | Type | Length | Decimal | Domain
PARTITION | SMALLINT | 4 | 0 | Partition number; 0 if table space is not partitioned
TSNAME | VARCHAR | 24 | 0 | Name of the table space
DBNAME | VARCHAR | 24 | 0 | Name of the database that contains the table space
IXNAME | VARCHAR | 128 | 0 | Name of the partitioning index
IXCREATOR | VARCHAR | 128 | 0 | The schema of the partitioning index
PQTY | INTEGER | 4 | 0 | Primary quantity
SQTY | SMALLINT | 4 | 0 | Secondary quantity
STORTYPE | CHAR | 1 | 0 | Type of storage allocation
STORNAME | VARCHAR | 128 | 0 | Name of storage group used for space allocation
VCATNAME | VARCHAR | 24 | 0 | Name of integrated catalog facility catalog used for space allocation
CARD | INTEGER | 4 | 0 | Number of rows in the table space or partition
FARINDREF | INTEGER | 4 | 0 | Number of rows that have been relocated far from their original page
NEARINDREF | INTEGER | 4 | 0 | Number of rows that have been relocated near their original page
PERCACTIVE | SMALLINT | 4 | 0 | Percentage of space occupied by rows of data from active tables
PERCDROP | SMALLINT | 4 | 0 | Percentage of space occupied by rows of dropped tables
IBMREQD | CHAR | 1 | 0 |
LIMITKEY | VARCHAR | 512 | 0 | The high value of the partition in external format
FREEPAGE | SMALLINT | 4 | 0 | Number of pages loaded before a page is left as free space
PCTFREE | SMALLINT | 4 | 0 | Percentage of each page left as free space
CHECKFLAG | CHAR | 1 | 0 |
CHECKRID | CHAR | 4 | 0 |
SPACE | INTEGER | 4 | 0 |
COMPRESS | CHAR | 1 | 0 |
PAGESAVE | SMALLINT | 4 | 0 |
STATSTIME | TIMESTAMP | 26 | 0 | RUNSTATS updated the statistics
GBPCACHE | CHAR | 1 | 0 | Group buffer pool cache option specified for this table space or table space partition
Parent topic: Table and Column Descriptions


HSSYSTSP

The HSSYTSP table is used for SYSTABLESPACE views. The following table describes the columns in the HSSYSTSP table:

Column | Type | Length | Decimal | Domain
NAME | VARCHAR | 24 | 0 | Name of the table space
CREATOR | VARCHAR | 128 | 0 | Authorization ID of the owner of the table space
DBNAME | VARCHAR | 24 | 0 | Name of the database that contains the table space
DBID | SMALLINT | 4 | 0 | Internal identifier of the database which contains the table Space
OBID | SMALLINT | 4 | 0 | Internal identifier of the table space file descriptor
PSID | SMALLINT | 4 | 0 | Internal identifier of the table space page set descriptor
BPOOL | CHAR | 8 | 0 | Name of the buffer pool used for the table space
PARTITIONS | SMALLINT | 4 | 0 | Number of partitions of the table space; 0 if the table space is not partitioned
LOCKRULE | CHAR | 1 | 0 | Lock size of the table space
PGSIZE | SMALLINT | 4 | 0 | Size of pages in the table space in kilobytes
ERASERULE | CHAR | 1 | 0 | Whether the data sets are to be erased when dropped
STATUS | CHAR | 1 | 0 | Availability status of the table space
IMPLICIT | CHAR | 1 | 0 | Whether the table space was created implicitly
NTABLES | SMALLINT | 4 | 0 | Number of tables defined in the table space
NACTIVE | INTEGER | 4 | 0 | Number of active pages in the table space
DSETPASS | CHAR | 8 | 0 |
CLOSERULE | CHAR | 1 | 0 | Whether the data sets are candidates for closure when the limit on the number of open data sets is reached
SPACE | INTEGER | 4 | 0 |
IBMREQD | CHAR | 1 | 0 |
ROOTNAME | VARCHAR | 18 | 0 |
ROOTCREATOR | CHAR | 8 | 0 |
SEGSIZE | SMALLINT | 4 | 0 | Number of pages in each segment of a segmented table space
CREATEDBY | VARCHAR | 128 | 0 | Primary authorization ID of the user who created the table space
STATSTIME | TIMESTAMP | 26 | 0 | RUNSTATS updated the statistics
LOCKMAX | INTEGER | 4 | 0 | The maximum number of locks per user to acquire for the table or table space before escalating to the next locking level
TYPE | CHAR | 1 | 0 | The type of table space
Parent topic: Table and Column Descriptions


HSSYSVDE

The HSSYVDE table is used for SYSVIEWDEP views. The following table describes the columns in the HSSYSVDE table:

Column | Type | Length | Decimal | Domain
BNAME | VARCHAR | 128 | 0 | Name of the object on which the view is dependent
BCREATOR | VARCHAR | 128 | 0 | Authorization ID of the owner of BNAME
BTYPE | CHAR | 1 | 0 | Type of object
DNAME | VARCHAR | 128 | 0 | Name of the view.
DCREATOR | VARCHAR | 128 | 0 | The schema of the view
IBMREQD | CHAR | 1 | 0 |
BSCHEMA | VARCHAR | 128 | 0 | Schema of BNAME
Parent topic: Table and Column Descriptions


HSSYSVIE

The HSSYVIE table is used for SYSVIEWS views. The following table describes the columns in the HSSYSVIE table:

Column | Type | Length | Decimal | Domain
NAME | VARCHAR | 128 | 0 | Name of the object
CREATOR | VARCHAR | 128 | 0 | The schema of the object
SEQNO | SMALLINT | 4 | 0 | Sequence number of this row
CHECK | CHAR | 1 | 0 | Whether the WITH CHECK OPTION clause was specified in the CREATE VIEW statement
IBMREQD | CHAR | 1 | 0 |
TEXT | VARCHAR | 254 | 0 | Text or portion of the text of the statement that was used to create the object
Parent topic: Table and Column Descriptions


HSURDAPP

The HSURDAPP table is used for the application identifiers list, and is populated during the Life Cycle phase. The following table describes the columns in the HSURDAPP table:

Column | Type | Length | Decimal | Domain
APPLID | CHARACTER | 10 | 0 | application ID
DESAPP | CHARACTER | 50 | 0 | description
DATEINS | CHARACTER | 8 | 0 | insert date
TIMEINS | CHARACTER | 8 | 0 | insert time
Parent topic: Table and Column Descriptions


HSURDAUT

The HSURDAUT table is used for storing user profiling information, and is populated during the User Credentials phase:
Column | Type | Length | Decimal | Domain
MCRECID | BINARY | 4 | 0 | machine/company ID
AUTHUSER | CHARACTER | 128 | 0 | user name
AUTHPERMIT | CHARACTER | 128 | 0 | authority type
Parent topic: Table and Column Descriptions


HSURDCAR

The HSURDCAR table is used for DB2 table cardinality and is populated during the DB2 Catalog Synchronization phase. The following table describes the columns in the HSURDCAR table:

Column | Type | Length | Decimal | Domain
TBCREATOR | CHARACTER | 128 | 0 | table creator
TBNAME | CHARACTER | 128 | 0 | table name
CARD | BINARY | 4 | 0 | table cardinality
Parent topic: Table and Column Descriptions


HSURDCJD

The HSURDCJD table is used for storing custom jobs, and is populated during the Work with Jobs phase. The following table describes the columns in the HSURDCJD table:

Column | Type | Length | Decimal | Domain
JOBNAME | CHARACTER | 8 | 0 | job name
PARMPRG | BINARY | 4 | 0 | parameter progressive
PARMDESC | CHARACTER | 100 | 0 | parameter description
PARMVAR | CHARACTER | 10 | 0 | parameter variable name
PARMTYPE | CHARACTER | 1 | 0 | parameter type
PARMLEN | BINARY | 4 | 0 | parameter length
Parent topic: Table and Column Descriptions


HSURDCJH

The HSURDCJH table is used for storing custom jobs, and is populated during the Work with Jobs phase. The following table describes the columns in the HSURDCJH table:

Column | Type | Length | Decimal | Domain
JOBNAME | CHARACTER | 8 | 0 | job name
SKLNAME | CHARACTER | 8 | 0 | skeleton name
JOBDESC | CHARACTER | 100 | 0 | job description
JOBPROCESS | CHARACTER | 20 | 0 | job process
PARMNUM | BINARY | 4 | 0 | job parameter number
Parent topic: Table and Column Descriptions


HSURDCLA

The HSURDCLA table is used for the field classes list and is populated during the Work with Field Class phase. The following table describes the columns in the HSURDCLA table:

Column | Type | Length | Decimal | Domain
CLATYPE | BINARY | 3 | 0 | class identifier
CLACLS | CHARACTER | 1 | 0 | short class identifier
DSCLATYPE | CHARACTER | 30 | 0 | class description
DRCLATYPE | CHARACTER | 7 | 0 | class short description
EUROSENS | CHARACTER | 1 | 0 | euro sensitive flag
PRIMCLS | BINARY | 3 | 0 | primary class
FLALFA | CHARACTER | 1 | 0 | alphanumeric field
FLZND | CHARACTER | 1 | 0 | zoned field
FLPCK | CHARACTER | 1 | 0 | packed field
FLBIN | CHARACTER | 1 | 0 | binary field
FLEDT | CHARACTER | 1 | 0 | edited field
LENMIN | BINARY | 5 | 0 | minimum length
LENMAX | BINARY | 5 | 0 | maximum length
INTMIN | BINARY | 3 | 0 | minimum integer
INTMAX | BINARY | 3 | 0 | maximum integer
DECMIN | BINARY | 3 | 0 | minimum decimal
DECMAX | BINARY | 3 | 0 | maximum decimal
SUPCLASS | CHARACTER | 1 | 0 | super class
EDITMSK | CHARACTER | 32 | 0 | edit mask
FLCARD | CHARACTER | 3 | 0 | cardinality
Parent topic: Table and Column Descriptions


HSURDCOL

The HSURDCOL table is used for SYSCOLUMNS views. The following table describes the columns in the HSURDCOL table:

Column | Type | Length | Decimal | Domain
NAME | CHARACTER | 128 | 0 | field name
TBNAME | CHARACTER | 128 | 0 | table name
TBCREATOR | CHARACTER | 128 | 0 | owner of the table
COLNO | BINARY | 4 | 0 | number of column
COLTYPE | CHARACTER | 8 | 0 | field type
LENGTH | BINARY | 4 | 0 | field length
SCALE | BINARY | 4 | 0 | field scale
NULLS | CHARACTER | 1 | 0 | null flag
REMARKS | CHARACTER | 254 | 0 | field description
HIDDEN | CHARACTER | 1 | 0 | hidden column (for internal use only)
Parent topic: Table and Column Descriptions


HSURDCOM

The HSURDCOM table is used for the companies list and is populated during the Work with Company phase. The following table describes the columns in the HSURDCOM table:

Column | Type | Length | Decimal | Domain
MACHINEID | CHARACTER | 10 | 0 | machine ID
CDSOC | CHARACTER | 10 | 0 | company code
MCRECID | BINARY | 4 | 0 | machine/company ID
DESCSOC | CHARACTER | 50 | 0 | description
DATEINS | CHARACTER | 8 | 0 | insert date
TIMEINS | CHARACTER | 8 | 0 | insert time
Parent topic: Table and Column Descriptions


HSURDCPY

The HSURDCPY table is used for the copybook/includes list and is populated during the Work with Copybook phase. The following table describes the columns in the HSURDCPY table:

Column | Type | Length | Decimal | Domain
MCRECID | BINARY | 4 | 0 | machine/company ID
CPYLIB | CHARACTER | 44 | 0 | copybook library
CPYNAME | CHARACTER | 10 | 0 | copybook name
IOAREANAME | CHARACTER | 30 | 0 | I/O area name
IOAREAPRG | BINARY | 3 | 0 | I/O area progressive
CPYTXT | CHARACTER | 50 | 0 | copybook text
FLANAL | CHARACTER | 1 | 0 | analysis flag
FLBAD | CHARACTER | 1 | 0 | bad copybook flag
FLBADTYPE | CHARACTER | 1 | 0 | type of anomaly
LANGUAGE | CHARACTER | 3 | 0 | language
FLMORE01 | CHARACTER | 1 | 0 | flag more than one 01 level
FLMOREPRG | CHARACTER | 1 | 0 | flag more than one progressive
ASSOCIATED | CHARACTER | 1 | 0 | I/O area associated
RECLEN | BINARY | 5 | 0 | record length
NUMFLD | BINARY | 5 | 0 | number of fields
NUMFLDPCK | BINARY | 5 | 0 | number of packed fields
NUMFLDBIN | BINARY | 5 | 0 | number of binary fields
Parent topic: Table and Column Descriptions


HSURDDFT

The HSURDDFT table is used for analysis parameters and is populated during the Define Copybook Analysis Parameters phase and Define File Analysis Parameters phase. The following table describes the columns in the HSURDDFT table:

Column | Type | Length | Decimal | Domain
STEP | CHARACTER | 1 | 0 | step value
MCRECID | BINARY | 4 | 0 | machine/company ID
APPLID | CHARACTER | 10 | 0 | application ID
FILRECID | BINARY | 9 | 0 | file ID
FLDRECID | BINARY | 9 | 0 | field ID
PARMNAME | CHARACTER | 10 | 0 | parameter name
VALUE | CHARACTER | 10 | 0 | value
LENVALUE | CHARACTER | 2 | 0 | length of VALUE field
Parent topic: Table and Column Descriptions


HSURDEKY

The HSURDEKY table is used for DB2 table external keys and is populated during the DB2 Catalog Synchronization phase. The following table describes the columns in the HSURDEKY table:

Column | Type | Length | Decimal | Domain
TBCREATOR | CHARACTER | 128 | 0 | table creator
TBNAME | CHARACTER | 128 | 0 | table name
IXCREATOR | CHARACTER | 128 | 0 | index creator
IXNAME | CHARACTER | 128 | 0 | index name
COLSEQ | BINARY | 2 | 0 | index column sequence
COLNAME | CHARACTER | 30 | 0 | index column name
COLNO | BINARY | 2 | 0 | index column number
Parent topic: Table and Column Descriptions


HSURDEXT

The HSURDEXT table is used for extention file fields and is populated during the Define Company phase. The following table describes the columns in the HSURDEXT table:

Column | Type | Length | Decimal | Domain
MCRECID | BINARY | 4 | 0 | machine/company ID
FILRECID | BINARY | 9 | 0 | copybook library
FLDRECID | BINARY | 9 | 0 | copybook name
PCLASS | CHARACTER | 10 | 0 | I/O area name
PNAME | CHARACTER | 240 | 0 | I/O area progressive
PPRG | BINARY | 4 | 0 | copybook text
PVALNAT | CHARACTER | 1 | 0 | analysis flag
PVALLEN | BINARY | 5 | 0 | bad copybook flag
PVALDEC | BINARY | 3 | 0 | type of anomaly
PVALUE | CHARACTER | 254 | 0 | language
PEXTVALUE | VARCHAR | 1024 | 0 | flag more than one 01 level
PUPDATE | TIMESTAMP |  | 0 | flag more than one progressive
Parent topic: Table and Column Descriptions


HSURDFIL

The HSURDFIL table is used for the file list and is populated during the Load Data Store Information phase and Work with Data Store phase. The following table describes the columns in the HSURDFIL table:

Column | Type | Length | Decimal | Domain
MCRECID | BINARY | 4 | 0 | machine/company ID
FILLIB | CHARACTER | 128 | 0 | file library
FILTYPE | CHARACTER | 4 | 0 | file type
FILNAME | CHARACTER | 128 | 0 | file name
FILVER | BINARY | 3 | 0 | file version
RECFMT | CHARACTER | 10 | 0 | format of record
FILRECID | BINARY | 9 | 0 | file ID
TSNAME | CHARACTER | 24 | 0 | tablespace name
DBNAME | CHARACTER | 24 | 0 | database name
OBID | BINARY | 4 | 0 | OBID
APPLID | CHARACTER | 10 | 0 | application ID
FILTXT | CHARACTER | 50 | 0 | text
FREQLOG | CHARACTER | 1 | 0 | logging frequency
FREQSPE | CHARACTER | 1 | 0 | sampling frequency
FREQCORR | CHARACTER | 1 | 0 | data corruption frequency
NUMREC | BINARY | 9 | 0 | number of records
PROCID | CHARACTER | 5 | 0 | process ID
UNLFILTYP | CHARACTER | 4 | 0 | unload file type
UNLFILNAM | CHARACTER | 128 | 0 | unload file name
UNLFILVER | BINARY | 3 | 0 | unload file version
RECLEN | BINARY | 5 | 0 | length of record
FMTSEL | CHARACTER | 8 | 0 | format selector
DATEANA | CHARACTER | 8 | 0 | date of analysis
DATECAL | CHARACTER | 8 | 0 | date of last calculation
DATELOG | CHARACTER | 8 | 0 | date of last log
DATESPE | CHARACTER | 8 | 0 | date of last spectrum
DATECORR | CHARACTER | 8 | 0 | date of last data corruption
MBFTYP | CHARACTER | 1 | 0 | type of file
ANALYSIS | CHARACTER | 1 | 0 | analysis state
RECANAL | BINARY | 9 | 0 | number of record analyzed
CPYLIB | CHARACTER | 44 | 0 | associated copybook PDS
CPYNAME | CHARACTER | 10 | 0 | associated copybook name
IOAREANAME | CHARACTER | 30 | 0 | I/O area name
IOAREAPRG | BINARY | 3 | 0 | I/O area progressive
INFOPROV | CHARACTER | 3 | 0 | information provider
FILNICK | CHARACTER | 10 | 0 | nickname of file
LOGMON | CHARACTER | 1 | 0 | logging enabled
DCMON | CHARACTER | 1 | 0 | data corruption enabled
LOGALARM | CHARACTER | 1 | 0 | logging alarm
DCALARM | CHARACTER | 1 | 0 | data corruption alarm
FLANAG | CHARACTER | 1 | 0 | is demographic
COENAB | CHARACTER | 1 | 0 | data consistency
DATECO | CHARACTER | 8 | 0 | consistency date
FANAGPRO | CHARACTER | 3 | 0 | demographic provider
FILCAT | BINARY | 4 | 0 | file category
FLAG01 | CHARACTER | 1 | 0 | is demographic
FLAG02 | CHARACTER | 1 | 0 | is demographic
FLAG03 | CHARACTER | 1 | 0 | is demographic
FLAG04 | CHARACTER | 1 | 0 | is demographic
FLAG05 | CHARACTER | 1 | 0 | is demographic
TYPNAME | CHARACTER | 128 | 0 | ODBC Database Type Name
Parent topic: Table and Column Descriptions


HSURDFIN

The HSURDFIN table is used for Oracle nested tables and is populated during the Distributed loader phase. The following table describes the columns in the HSURDFIN table:
Column | Type | Length | Decimal | Domain
MCRECID | BINARY | 4 | 0 | machine/company ID
FILLIB | CHARACTER | 128 | 0 | file library
FILTYPE | CHARACTER | 4 | 0 | file type
FILNAME | CHARACTER | 128 | 0 | file name
FILRECID | BINARY | 9 | 0 | file ID
FILRECIDN | BINARY | 9 | 0 | file ID for nested
COLNAME | CHARACTER | 128 | 0 | column name
UNLFILTYP | CHARACTER | 4 | 0 | unload file type
UNLFILNAM | CHARACTER | 128 | 0 | unload file name
TYPNAME | CHAR | 128 | 0 | ODBC Data Type Name
Parent topic: Table and Column Descriptions


HSURDINT

The HSURDINT table is used to store interfaces that manage metadata import and export, and is populated during the export phase, or populated manually. The following table describes the columns in the HSURDINT table:

Column | Type | Length | Decimal | Domain
INTNAME | CHARACTER | 10 | 0 | Interface Name
INTTYPE | CHARACTER | 3 | 0 | Interface Type
INTPRG | BINARY | 9 | 0 | Interface Program
INTTXT | VARCHAR | 4000 | 0 | Interface Text
Parent topic: Table and Column Descriptions


HSURDICP

The HSURDICP table is used for image copybook data set names and is populated during the DB2 Catalog Synchronization phase. The following table describes the columns in the HSURDICP table:

Column | Type | Length | Decimal | Domain
DBNAME | CHARACTER | 24 | 0 | database name
TSNAME | CHARACTER | 24 | 0 | tablespace name
DSNUM | INTEGER | 4 | 0 | data set number
ICDATE | CHARACTER | 8 | 0 | date
DSNAME | CHARACTER | 44 | 0 | data set name
DSTIMESTAMP | CHARACTER | 26 | 0 | date and time
Parent topic: Table and Column Descriptions


HSURDJOB

The HSURDJOB table is used for scheduled client jobs and is populated during the Client – Work with Jobs phase. The following table describes the columns in the HSURDJOB table:

Column | Type | Length | Decimal | Domain
JOBRECID | BINARY | 9 | 0 | job ID
USERID | CHARACTER | 10 | 0 | user ID
JOBNAME | CHARACTER | 10 | 0 | job name
JOBPARM | CHARACTER | 1024 | 0 | job parameter
FLSTATUS | CHARACTER | 1 | 0 | flag status
DTTMSUBM | CHARACTER | 16 | 0 | scheduled timestamp
DTTMSTART | CHARACTER | 16 | 0 | submitted timestamp
DTTMEND | CHARACTER | 16 | 0 | ended timestamp
Parent topic: Table and Column Descriptions


HSURDKEY

The HSURDKEY table is used for DB2 table keys and is populated during the DB2 Catalog Synchronization phase. The following table describes the columns in the HSURDKEY table:

Column | Type | Length | Decimal | Domain
TBCREATOR | CHARACTER | 128 | 0 | table creator
TBNAME | CHARACTER | 128 | 0 | table name
IXCREATOR | CHARACTER | 128 | 0 | index creator
IXNAME | CHARACTER | 128 | 0 | index name
COLSEQ | BINARY | 2 | 0 | index column sequence
COLNAME | CHARACTER | 30 | 0 | index column name
COLNO | BINARY | 2 | 0 | index column number
Parent topic: Table and Column Descriptions


HSURDLIC

The HSURDLIC table is used for DB2 table information and is populated during the phase one of the Life Cycle. The following table describes the columns in the HSURDLIC table:

Column | Type | Length | Decimal | Domain
FILLIB | CHARACTER | 128 | 0 | file library
FILNAME | CHARACTER | 128 | 0 | file name
TSNAME | CHARACTER | 24 | 0 | tablespace name
DBNAME | CHARACTER | 24 | 0 | database name
OBID | SMALLINT | 4 | 0 | OBID
FILTXT | CHARACTER | 50 | 0 | text
NUMREC | INTEGER | 9 | 0 | number of records
RECLEN | INTEGER | 5 | 0 | length of record
MCRECID | SMALLINT | 4 | 0 | machine/company ID
FILRECID | INTEGER | 9 | 0 | file ID
ODBNAME | CHARACTER | 128 | 0 | ODBC entry
TYPNAME | CHARACTER | 128 | 0 | ODBC database type name
NEWLVLID | CHARACTER | 24 | 0 | new level number
Parent topic: Table and Column Descriptions


HSURDLOG

The HSURDLOG table is used for error messages and is populated during all batch functions. The following table describes the columns in the HSURDLOG table:

Column | Type | Length | Decimal | Domain
MACHINEID | CHARACTER | 10 | 0 | machine ID
CDSOC | CHARACTER | 10 | 0 | company
PGMNAME | CHARACTER | 10 | 0 | program name
JOBNAME | CHARACTER | 10 | 0 | job name
USERNAME | CHARACTER | 10 | 0 | user name
JOBID | INTEGER |  |  | job ID
TIMESTMP | CHARACTER | 26 | 0 | timestamp
PROGR | CHARACTER | 9 | 0 | progressive number
TPERR | CHARACTER | 1 | 0 | error type
TPSRC | CHARACTER | 3 | 0 | source type
CPYLIB | CHARACTER | 44 | 0 | copybook library
CPYNAME | CHARACTER | 10 | 0 | copybook name
FILLIB | CHARACTER | 128 | 0 | file library
FILTYPE | CHARACTER | 4 | 0 | file type
FILNAME | CHARACTER | 128 | 0 | file name
FILVER | CHARACTER | 3 | 0 | file version
RECFMT | CHARACTER | 10 | 0 | format of record
FILRECID | INTEGER |  |  | file ID
FLDNAME | CHARACTER | 128 | 0 | field name
FLDDISPL | INTEGER |  | 0 | Field displacement
RIGA | CHARACTER | 5 | 0 | row number
OBJ | CHARACTER | 10 | 0 | object name
TPOBJ | CHARACTER | 3 | 0 | object type
OPER | CHARACTER | 8 | 0 | operation
RC | CHARACTER | 2 | 0 | return dode
MSGID | CHARACTER | 7 | 0 | message ID
PARM | VARCHAR | 512 | 0 | message data field values
DATE | CHARACTER | 6 | 0 | date
TIME | CHARACTER | 8 | 0 | time
DESCR | CHARACTER | 132 | 0 | description
FLGPHASE | CHARACTER | 1 | 0 | phase flag
MSGTYPE | CHARACTER | 1 | 0 | message type
FCHECK | CHARACTER | 1 | 0 | flag check
METHOD | CHARACTER | 10 | 0 | method
CAUSE | VARCHAR | 512 | 0 | cause
SOLUTION | VARCHAR | 512 | 0 | solution
RESULT | VARCHAR | 512 | 0 | result
Parent topic: Table and Column Descriptions


HSURDMF

The HSURDMF table is used for the messages list and is populated during Data Express setup. The following table describes the columns in the HSURDMF table:

Column | Type | Length | Decimal | Domain
MSGID | CHARACTER | 7 | 0 | message identifier
MSG | CHARACTER | 132 | 0 | first-level message text
SEV | CHARACTER | 2 | 0 | severity code
FLCHG | CHARACTER | 1 | 0 | flag changed
LSTUSR | CHARACTER | 10 | 0 | user
SECLVL | VARCHAR | 3000 | 0 | second-level message text
Parent topic: Table and Column Descriptions


HSURDMFD

The HSURDMFD table is used for the messages parameter list and is populated during Data Express setup. The following table describes the columns in the HSURDMFD table:

Column | Type | Length | Decimal | Domain
MSGID | CHARACTER | 7 | 0 | message identifier
PRGDTL | DECIMAL | 5 | 0 | progressive detail
FMTTYPE | CHARACTER | 8 | 0 | format data type
FMTLEN | DECIMAL | 5 | 0 | format length
FMTDEC | DECIMAL | 2 | 0 | format decimal position
Parent topic: Table and Column Descriptions


HSURDMID

The HSURDMID table is used for the machine identifiers list and is populated during the Work with Machine Identifier phase. The following table describes the columns in the HSURDMID table:

Column | Type | Length | Decimal | Domain
MACHINEID | CHARACTER | 10 | 0 | machine identifier
DESMID | CHARACTER | 50 | 0 | description
MODEL | CHARACTER | 10 | 0 | model
PROC | CHARACTER | 3 | 0 | processor
DATEINS | CHARACTER | 8 | 0 | insert date
TIMEINS | CHARACTER | 8 | 0 | insert time
Parent topic: Table and Column Descriptions


HSURDNCC

The HSURDNCC table is used for the field class naming convention and is populated during the Work with Field Class phase. The following table describes the columns in the HSURDNCC table:

Column | Type | Length | Decimal | Domain
CLATYPE | BINARY | 3 | 0 | class Identifier
PROG | BINARY | 3 | 0 | progressive number
INCFLAG | CHARACTER | 1 | 0 | include/exclude flag
VALUE | CHARACTER | 32 | 0 | string value
Parent topic: Table and Column Descriptions


HSURDPID

The HSURDPID table is used for the process identifiers list and is populated during the Work with Process Identifier phase. The following table describes the columns in the HSURDPID table:

Column | Type | Length | Decimal | Domain
PROCID | BINARY | 5 | 0 | process ID
ACCESS | CHARACTER | 1 | 0 | access type
RECFMT | CHARACTER | 2 | 0 | record format
RECLEN | BINARY | 5 | 0 | record length
TRKPOS | BINARY | 5 | 0 | TRK position
TRKLEN | BINARY | 5 | 0 | TRK length
PROCDES | CHARACTER | 30 | 0 | process ID description
IOPGMNAME | CHARACTER | 8 | 0 | related I/O routine
DATPOS | BINARY | 5 | 0 | data position
DATLENTYP | CHARACTER | 1 | 0 | Data length type
DATLENPOS | BINARY | 5 | 0 | data length position
DATLENLEN | BINARY | 5 | 0 | data length
Parent topic: Table and Column Descriptions


HSURDPRG

The HSURDPRG table is used for ISPF panel progressives and is populated during all ISPF functions.

Column | Type | Length | Decimal | Domain
TABPRG | DECIMAL | 6 | 0 | ISPF tables progressive
Parent topic: Table and Column Descriptions


HSURDPRO

The HSURDPRO table is used for enabling information and is populated during the Product Enabling phase. The following table describes the columns in the HSURDPRO table:

Column | Type | Length | Decimal | Domain
FIELD1 | CHARACTER | 24 | 0 | reserved
FIELD2 | CHARACTER | 96 | 0 | reserved
Parent topic: Table and Column Descriptions


HSURDROU

The HSURDROU table is used for storing routines documentation information, and is populated during the Work with Routines phase. The following table describes the columns in the HSURDROU table:

Column | Type | Length | Decimal | Domain
ROUTNAME | CHARACTER | 8 | 0 | routine name
DESROUT | CHARACTER | 50 | 0 | routine description
DATEINS | CHARACTER | 8 | 0 | routine creation date
TIMEINS | CHARACTER | 8 | 0 | routine creation time
ROUTYPE | CHARACTER | 1 | 0 | routine type
Parent topic: Table and Column Descriptions


HSURDSYC

The HSURDSYC table is used for SYSTABLESPACE views. The following table describes the columns in the HSURDSYC table:

Column | Type | Length | Decimal | Domain
DBNAME | CHAR | 24 | 0 | Name of the database
TSNAME | CHAR | 24 | 0 | Name of the target table space or index space
DSNUM | INTEGER | 4 | 0 | Data set number within the table space
ICTYPE | CHAR | 1 | 0 | Type of operation
ICDATE | CHAR | 6 | 0 |
START_RBA | CHAR | 6 | 0 | A 48-bit positive integer that contains the LRSN of a point in the DB2 recovery log
FILESEQNO | INTEGER | 4 | 0 | Tape file sequence number of the copy
DEVTYPE | CHAR | 8 | 0 | Tape file sequence number of the copy
IBMREQD | CHAR | 1 | 0 |
DSNAME | CHAR | 44 | 0 | Contains the data set name. Otherwise, DSNAME contains the name of the database and table space or index space in the form
ICTIME | CHAR | 6 | 0 |
SHRLEVEL | CHAR | 1 | 0 | SHRLEVEL parameter on COPY
DSVOLSER | VARCHAR | 1784 | 0 | The volume serial numbers of the data set
TIMESTAMP | TIMESTAMP | 26 | 0 | The date and time when the row was inserted
ICBACKUP | CHAR | 2 | 0 | Specifies the type of image copy contained in the data set
ICUNIT | CHAR | 8 | 0 | Indicates the media that the image copy data set is stored on
STYPE | CHAR | 1 | 0 |
PIT_RBA | CHAR | 6 | 0 |
GROUP_MEMBER | CHAR | 8 | 0 | The DB2 data sharing member name of the DB2 subsystem that performed the operation
OTYPE | CHAR | 1 | 0 | Type of object that the recovery information is for
LOWDSNUM | INTEGER | 4 | 0 | Partition number of the lowest partition in the range for SYSCOPY records created for REORG and LOAD REPLACE for resetting a REORG pending status
HIGHDSNUM | INTEGER | 4 | 0 | Partition number of the highest partition in the range. This column is valid only for SYSCOPY records created for REORG and LOAD REPLACE for resetting REORG pending status
Parent topic: Table and Column Descriptions


HSURDSYT

The HSURDSYT table is used for SYSTABLES views. The following table describes the columns in the HSURDSYT table:

Column | Type | Length | Decimal | Domain
NAME | VARCHAR | 128 | 0 | Name of the table
CREATOR | VARCHAR | 128 | 0 | Owner of the table
TYPE | CHAR | 1 | 0 | Type of object
COLCOUNT | SMALLINT | 4 | 0 | Number of columns
RECLENGTH | SMALLINT | 4 | 0 | Record length
LABEL | CHAR | 30 | 0 | Label of the table
CREATEDTS | TIMESTAMP | 26 | 0 | Time creation table
ALTEREDTS | TIMESTAMP | 26 | 0 | Time last alteration
TSNAME | CHAR | 24 | 0 | Table space name
DBNAME | CHAR | 24 | 0 | Data base name
OBID | SMALLINT | 4 | 0 | OBID
Parent topic: Table and Column Descriptions


HSURDUE

The HSURDUE table is used for user exits and is intended for future use. The following table describes the columns in the HSURDUE table:

Column | Type | Length | Decimal | Domain
USEREXIT | CHAR | 10 | 0 | user exit
PROGR | SMALLINT | 2 | 0 | progressive
PARMNAME | CHAR | 10 | 0 | parameter name
PARMVAL | CHAR | 80 | 0 | parameter value
Parent topic: Table and Column Descriptions


HSURDUEV

The HSURDUEV table is used for user exit additional fields and is intended for future use. The following table describes the columns in the HSURDUEV table:

Column | Type | Length | Decimal | Domain
MCRECID | SMALLINT | 2 | 0 | machine/company ID
FILRECID | INTEGER | 4 | 0 | file ID
FLDRECID | INTEGER | 4 | 0 | field ID
CLASS | SMALLINT | 2 | 0 | class
USEREXIT | CHAR | 10 | 0 | user exit
PROGR | SMALLINT | 2 | 0 | parameter progressive
FLDADD | INTEGER | 4 | 0 | additional field ID
CLAADD | SMALLINT | 2 | 0 | additional class
CONST | VARCHAR | 256 | 0 | constant
Parent topic: Table and Column Descriptions


HSURDUSR

The HSURDUSR table is used for storing user profiling information, and is populated during the User Credentials phase:
Column | Type | Length | Decimal | Domain
AUTHUSER | CHARACTER | 128 | 0 | user name
AUTHPASSWORD | CHARACTER | 128 | 0 | user password
DBUSER | CHARACTER | 128 | 0 | DB user name
DBPASSWORD | CHARACTER | 128 | 0 | DB user password
Parent topic: Table and Column Descriptions


MSURDLOG

The MSURDLOG table is currently unused, but is present to maintain compatibility with versions of Data Express earlier than version 4.0 Update 24. The following table describes the columns in the MSURDLOG table:

Column | Type | Length | Decimal | Domain
MACHINEID | CHARACTER | 10 | 0 | Machine ID
CDSOC | CHARACTER | 10 | 0 | Company
PGM | CHARACTER | 10 | 0 | Program Name
PROGR | CHARACTER | 2 | 0 | Progressive Number
TPERR | CHARACTER | 1 | 0 | Error Type
TYPE | CHARACTER | 3 | 0 | Source Type
CPYLIB | CHARACTER | 44 | 0 | Copy Library
CPYNAME | CHARACTER | 10 | 0 | Copy Name
FILLIB | CHARACTER | 44 | 0 | File Library
FILTYPE | CHARACTER | 4 | 0 | File Type
FILNAME | CHARACTER | 44 | 0 | File Name
FILVER | CHARACTER | 3 | 0 | File Version
RECFMT | CHARACTER | 10 | 0 | Format of Record
RIGA | CHARACTER | 5 | 0 | Row Number
OBJ | CHARACTER | 10 | 0 | Object Name
TPOBJ | CHARACTER | 3 | 0 | Object Type
OPER | CHARACTER | 8 | 0 | Operation
RC | CHARACTER | 2 | 0 | Return Code
MSGID | CHARACTER | 7 | 0 | Message Id
PARM | CHARACTER | 128 | 0 | Message Data Field Values
DATE | CHARACTER | 6 | 0 | Date
TIME | CHARACTER | 8 | 0 | Time
DESCR | CHARACTER | 132 | 0 | Description
FLGPHASE | CHARACTER | 1 | 0 | Phase Flag
Parent topic: Table and Column Descriptions


USDCHADD

The USDCHADD table is used for addresses and is populated during Data Express setup. The following table describes the columns in the USDCHADD table:

Column | Type | Length | Decimal | Domain
PRGADDR | BINARY | 4 | 0 | address progressive
ADDRESS | CHARACTER | 50 | 0 | address
Parent topic: Table and Column Descriptions


USDCHCOM

The USDCHCOM table is used for company titles and is populated during Data Express setup. The following table describes the columns in the USDCHCOM table:

Column | Type | Length | Decimal | Domain
PRGCOMR | BINARY | 4 | 0 | compny title progressive
COMPANY | CHARACTER | 50 | 0 | company title
Parent topic: Table and Column Descriptions


USDCHEMA

The USDCHEMA table is used for email addresses and is populated during Data Express setup. The following table describes the columns in the USDCHEMA table:

Column | Type | Length | Decimal | Domain
PRGADDR | BINARY | 4 | 0 | progressive email address
EMAILADD | CHARACTER | 50 | 0 | email address
Parent topic: Table and Column Descriptions


USDCHNAC

The USDCHNAC table is used for email addresses and is populated during Data Express setup. The following table describes the columns in the USDCHNAC table:

Column | Type | Length | Decimal | Domain
PRGNAME | BINARY | 4 | 0 | progressive name
PRGNAME2 | BINARY | 4 | 0 | second progressive name
SEX | CHARACTER | 1 | 0 | gender flag
NAME | CHARACTER | 50 | 0 | name
Parent topic: Table and Column Descriptions


USDCHNAM

The USDCHNAM table is used for names and is populated during Data Express setup. The following table describes the columns in the USDCHNAM table:

Column | Type | Length | Decimal | Domain
PRGNAME | BINARY | 4 | 0 | progressive name
NAME | CHARACTER | 50 | 0 | name
Parent topic: Table and Column Descriptions


USDCHSUC

The USDCHSUC table is used for email addresses and is populated during Data Express setup. The following table describes the columns in the USDCHSUC table:

Column | Type | Length | Decimal | Domain
PRGSURN | BINARY | 4 | 0 | progressive surname
PRGNAME2 | BINARY | 4 | 0 | second progressive surname
SURNAME | CHARACTER | 50 | 0 | name
Parent topic: Table and Column Descriptions


USDCHSUR

The USDCHSUR table is used for surnames and is populated during Data Express setup. The following table describes the columns in the USDCHSUR table:

Column | Type | Length | Decimal | Domain
PRGSURN | BINARY | 4 | 0 | surname progressive
SURNAME | CHARACTER | 50 | 0 | surname
Parent topic: Table and Column Descriptions


USURDCBE

The USURDCBE table is used for advanced features of the linked field lists and is populated during the Defined Combined Fields phase:

Column | Type | Length | Decimal | Domain
MCRECID | BINARY | 4 | 0 | machine/company ID
FILRECID | BINARY | 9 | 0 | file ID
CODCBZ | BINARY | 9 | 0 | conditional fields group
PRGCBZ | BINARY | 9 | 0 | progressive fields group
FLDRECID | CHARACTER | 1 | 0 | field ID
EXTTYPE | BINARY | 9 | 0 | extended value type
EXTPRG | BINARY | 9 | 0 | extended value progressive
EXTPOS | BINARY | 9 | 0 | position for substring
EXTLEN | CHARACTER | 100 | 0 | length for substring
EXTVAL | CHARACTER | 1 | 0 | prefix and suffix value
EXTHEX | CHARACTER | 8 | 0 | hexadecimal value
EXTROUTINE | BINARY | 9 | 0 | routine name
Parent topic: Table and Column Descriptions


USURDCBZ

The USURDCBZ table is used for the linked fields list and is populated during the Defined Combined Fields phase. The following table describes the columns in the USURDCBZ table:

Column | Type | Length | Decimal | Domain
MCRECID | BINARY | 4 | 0 | machine/company ID
FILRECID | BINARY | 9 | 0 | field ID
CODCBZ | BINARY | 5 | 0 | cod. fields group
PRGCBZ | BINARY | 5 | 0 | progressive fields group
CBZTXT | CHARACTER | 50 | 0 | fields group text
FLDRECID | BINARY | 9 | 0 | field ID
CTRLTAB | CHARACTER | 18 | 0 | control table
CTRLPGM | CHARACTER | 10 | 0 | control program
EDTSIG | CHARACTER | 1 | 0 | flag sign on string
EDTDEC | CHARACTER | 1 | 0 | flag decimal on string
Parent topic: Table and Column Descriptions


USURDDBM

The USURDDBM table is used for Oracle database mappings and is populated during the Oracle Catalog Mapping phase. The following table describes the columns in the USURDDBM table:

Column | Type | Length | Decimal | Domain
DBMID | INTEGER | 4 | 0 | mapping ID
SRCDBNAM | CHAR | 128 | 0 | source database name
SRCSCHEM | CHAR | 128 | 0 | source schema
SRCUSR | CHAR | 128 | 0 | source user
SRCPWD | CHAR | 128 | 0 | source password
TRGDBNAM | CHAR | 128 | 0 | target database name
TRGSCHEM | CHAR | 128 | 0 | target schema
TRGUSR | CHAR | 128 | 0 | target user
TRGPWD | CHAR | 128 | 0 | target password
FL01 | CHAR | 1 | 0 | flag 1
FL02 | CHAR | 1 | 0 | flag 2
FL03 | CHAR | 1 | 0 | flag 3
FL04 | CHAR | 1 | 0 | flag 4
MCRECID | BINARY | 4 | 0 | machine/company ID
GROUPNAME | CHARACTER | 10 | 0 | group name
METHOD | CHARACTER | 10 | 0 | method name
Parent topic: Table and Column Descriptions


USURDFLC

The USURDFLC table is used for user field information and is populated during the Analyze Copybook phase. The following table describes the columns in the USURDFLC table:

Column | Type | Length | Decimal | Domain
MCRECID | BINARY | 4 | 0 | machine/company ID
CPYLIB | CHARACTER | 44 | 0 | copybook library
CPYNAME | CHARACTER | 10 | 0 | copybook name
TRK | CHARACTER | 3 | 0 | record type
IOAREANAME | CHARACTER | 30 | 0 | I/O area name
IOAREAPRG | BINARY | 3 | 0 | I/O area progressive
IOAREASCP | BINARY | 3 | 0 | I/O area Scope
IOAREANBR | BINARY | 5 | 0 | I/O area number
FLDNBR | BINARY | 5 | 0 | field number
FLDNAME | CHARACTER | 128 | 0 | field name
FLDSCP | BINARY | 3 | 0 | field scope
FLDDISPL | BINARY | 5 | 0 | field displacement
STATUS | CHARACTER | 3 | 0 | information provider
LGLREDEF | CHARACTER | 1 | 0 | REDEFINES association type
LGLASSOC | BINARY | 5 | 0 | associated field number
Parent topic: Table and Column Descriptions


USURDIMP

The USURDIMP table is used for the VSAM import list and is populated during the Analyze Sources phase. The following table describes the columns in the USURDIMP table:

Column | Type | Length | Decimal | Domain
MACHINEID | CHARACTER | 10 | 0 | machine ID
CDSOC | CHARACTER | 10 | 0 | company code
MCRECID | BINARY | 04 | 0 | machine/company ID
TYPE | CHARACTER | 04 | 0 | file type
DBDNAME | CHARACTER | 08 | 0 | DBD name
FILENAME | CHARACTER | 54 | 0 | file name
APPLID | CHARACTER | 10 | 0 | application ID
FILEUNLO | CHARACTER | 50 | 0 | unload file name
FMTSEL | CHARACTER | 10 | 0 | format selector
NICKNAME | CHARACTER | 10 | 0 | nickname
FLFUNC | CHARACTER | 01 | 0 | function flag
NUMREC | BINARY | 09 | 0 | number of records
RECLEN | BINARY | 09 | 0 | length of records
SRCMCRID | BINARY | 04 | 0 | source machine/company ID
PDSNAME | CHARACTER | 044 | 0 | PDS name
SRCNAME | CHARACTER | 010 |  | source name
Parent topic: Table and Column Descriptions


USURDRIR

The USURDRIR table is used for referential Integrity process. The following table describes the columns in the USURDRIR table:

Column | Type | Length | Decimal | Domain
MCRECID | BINARY | 4 | 0 | machine/company ID
DBNAME | CHARACTER | 128 | 0 | Database Name
PSNAME | CHARACTER | 128 | 0 | Primary Key Schema Name
PTNAME | CHARACTER | 128 | 0 | Primary Key Table Name
PKNAME | CHARACTER | 128 | 0 | Primary Key Name
PKCOLNAM | CHARACTER | 128 | 0 | Field included in Primary Key
KCOLSEQ | BINARY | 4 | 0 | Field sequence in Primary Key
KUPRULE | BINARY | 4 | 0 | Primary Key Update Rule
KDELRULE | BINARY | 4 | 0 | Primary Key Delete Rule
FSNAME | CHARACTER | 128 | 0 | Foreign Key Schema Name
FTNAME | CHARACTER | 128 | 0 | Foreign Key Table Name
FKNAME | CHARACTER | 128 | 0 | Foreign Key Name
FKCOLNAM | CHARACTER | 128 | 0 | Field Included In Foreign Key
Parent topic: Table and Column Descriptions


V1DCHFIL

The V1DCHFIL table is used for HSDCHFIL and HSURDFIL views. The following table describes the columns in the V1CHFIL table:

Column | Type | Length | Decimal | Domain
MCRECID | BINARY | 4 | 0 | machine/company ID
FILLIB | CHARACTER | 128 | 0 | file library
FILTYPE | CHARACTER | 4 | 0 | file type
TYPNAME | CHARACTER | 128 | 0 | ODBC Database Type Name
FILNAME | CHARACTER | 128 | 0 | file name
FILVER | BINARY | 3 | 0 | file version
RECFMT | CHARACTER | 10 | 0 | format of record
FILRECID | BINARY | 9 | 0 | file ID
TSNAME | CHARACTER | 24 | 0 | tablespace name
DBNAME | CHARACTER | 24 | 0 | database name
OBID | BINARY | 4 | 0 | OBID
APPLID | CHARACTER | 10 | 0 | application ID
FILTXT | CHARACTER | 50 | 0 | text
FREQLOG | CHARACTER | 1 | 0 | logging frequency
FREQSPE | CHARACTER | 1 | 0 | sampling frequency
FREQCORR | CHARACTER | 1 | 0 | data corruption frequency
NUMREC | BINARY | 9 | 0 | number of records
PROCID | CHARACTER | 5 | 0 | process ID
UNLFILTYP | CHARACTER | 4 | 0 | unload file type
UNLFILNAM | CHARACTER | 44 | 0 | unload file name
UNLFILVER | BINARY | 3 | 0 | unload file version
RECLEN | BINARY | 5 | 0 | length of record
FMTSEL | CHARACTER | 8 | 0 | format selector
DATEANA | CHARACTER | 8 | 0 | date of analysis
DATECAL | CHARACTER | 8 | 0 | date of last calculation
DATELOG | CHARACTER | 8 | 0 | date of last log
DATESPE | CHARACTER | 8 | 0 | date of last spectrum
DATECORR | CHARACTER | 8 | 0 | date of last data corruption
MBFTYP | CHARACTER | 1 | 0 | type of file
ANALYSIS | CHARACTER | 1 | 0 | analysis state
RECANAL | BINARY | 9 | 0 | number of record analyzed
CPYLIB | CHARACTER | 44 | 0 | associated Copybook PDS
CPYNAME | CHARACTER | 10 | 0 | associated Copybook name
IOAREANAME | CHARACTER | 30 | 0 | I/O area name
IOAREAPRG | BINARY | 3 | 0 | I/O area progressive
INFOPROV | CHARACTER | 3 | 0 | information provider
FILNICK | CHARACTER | 10 | 0 | nickname of file
LOGMON | CHARACTER | 1 | 0 | Logging enabled
DCMON | CHARACTER | 1 | 0 | data corruption enabled
LOGALARM | CHARACTER | 1 | 0 | logging alarm
DCALARM | CHARACTER | 1 | 0 | data corruption alarm
FLANAG | CHARACTER | 1 | 0 | is demographic
COENAB | CHARACTER | 1 | 0 | data consistency
DATECO | CHARACTER | 8 | 0 | consistency date
FANAGPRO | CHARACTER | 3 | 0 | demographic provider
FILCAT | CHARACTER | 10 | 0 | file category
FLAG01 | CHARACTER | 1 | 0 | Flag 1
FLAG02 | CHARACTER | 1 | 0 | Flag 2
FLAG03 | CHARACTER | 1 | 0 | Flag 3
FLAG04 | CHARACTER | 1 | 0 | Flag 4
FLAG05 | CHARACTER | 1 | 0 | Flag 5
PROCIDINP | CHARACTER | 5 | 0 | process ID for input file
UNLINPTYP | CHARACTER | 4 | 0 | unload input file type
UNLINPNAM | CHARACTER | 44 | 0 | unload input file name
UNLINPVER | BINARY | 3 | 0 | unload input file version
FMTSEL1 | CHARACTER | 8 | 0 | format selector
SLTPGM | CHARACTER | 8 | 0 | select program
UNLOUTTYP | CHARACTER | 4 | 0 | unload output file type
UNLOUTNAM | CHARACTER | 44 | 0 | unload output file name
UNLOUTVER | BINARY | 3 | 0 | unload output file version
WRTPGM | CHARACTER | 8 | 0 | write program for output
METHOD | CHARACTER | 10 | 0 | Method
Parent topic: Table and Column Descriptions


V1DCHFLF

The V1DCHFLF table is used for ANURDFLF and ANDCHFLF views. The following table describes the columns in the V1DCHFLF table:

Column | Type | Length | Decimal | Domain
MCRECID | BINARY | 4 | 0 | machine/company ID
FILRECID | BINARY | 9 | 0 | file ID
FLDRECID | BINARY | 9 | 0 | field ID
FLDDISPL | BINARY | 5 | 0 | field displacement
FLDNAME | CHARACTER | 128 | 0 | field name
FLDLEN | BINARY | 5 | 0 | field length
FLDINT | BINARY | 3 | 0 | field integer
FLDDEC | BINARY | 3 | 0 | field decimal
FLDSIGN | CHARACTER | 1 | 0 | field sign
FLDTYPE CHARACTER 1 0

field type:

  • F = floating-point field
  • 1 = bit field
  • B = binary field
  • N = decimal field
  • P = packed-decimal field
  • A = alphanumeric field
  • E = edited field
TYPNAME | CHARACTER | 128 | 0 | ODBC Data Type Name
TYPNUM | DECIMAL | 5 | 0 | ODBC Data Type Number
FLDNAT | CHARACTER | 1 | 0 | field nature
FLJUSTIFY | CHARACTER | 1 | 0 | justify flag
CLATYPE | BINARY | 3 | 0 | field class
INFOPROV | CHARACTER | 3 | 0 | information origin
MINVAL | CHARACTER | 33 | 0 | minimum value
MAXVAL | CHARACTER | 33 | 0 | maximum value
AVGVAL | DECIMAL | 18 | 0 | average value
SPEDIFVAL | BINARY | 5 | 0 | number of different values
USERPGM | CHARACTER | 10 | 0 | user program
HASSPE | CHARACTER | 1 | 0 | spectrum presence flag
FLSPE | CHARACTER | 1 | 0 | spectrum calculation flag
LOGMON | CHARACTER | 1 | 0 | logging enabled
DCMON | CHARACTER | 1 | 0 | data corruption enabled
LOGALARM | CHARACTER | 1 | 0 | logging alarm
DCALARM | CHARACTER | 1 | 0 | data corruption alarm
NULLCAP | CHARACTER | 1 | 0 | null capable field
VARLEN | CHARACTER | 1 | 0 | field with variable length
ISANAG | CHARACTER | 1 | 0 | demographic field
ESTCARD | BINARY | 9 | 0 | estimated cardinality
ESTCLA | BINARY | 3 | 0 | estimated class
SAMPVAL | CHARACTER | 33 | 0 | sample value
SAMPNBR | BINARY | 9 | 0 | sample OCCURS
FLDTXT | CHARACTER | 50 | 0 | text
ISKEY | CHARACTER | 1 | 0 | primary key
FLCLAVAL | CHARACTER | 1 | 0 | use class value
ANAGPRO | CHARACTER | 3 | 0 | demographic provider
CHGFLG | CHARACTER | 1 | 0 | Data Changer active
CHGPGM | CHARACTER | 8 | 0 | Data Changer program
METHOD | CHARACTER | 10 | 0 | Method
Parent topic: Table and Column Descriptions


V1REP01

The V1REP01 table is used as a report that shows all classes defined and assigned to a data element. The following table describes the columns in the V1REP01 table:

Column | Type | Length | Decimal | Domain
CLASS_NUMBER | BINARY | 4 | 0 | Class, Number
CLASS_CODE | CHARACTER | 7 | 0 | Class Code
CLASS_DESCRIPTION | CHARACTER | 30 | 0 | Class Description
Parent topic: Table and Column Descriptions


V1REP02

The V1REP02 table is used as a report that shows all classes defined but not assigned to a data element. The following table describes the columns in the V1REP02 table:

Column | Type | Length | Decimal | Domain
CLASS_NUMBER | BINARY | 4 | 0 | Class Number
CLASS_CODE | CHARACTER | 7 | 0 | Class Code
CLASS_DESCRIPTION | CHARACTER | 30 | 0 | Class Description
Parent topic: Table and Column Descriptions


V1REP03

The V1REP03 table is used as a report that shows all tables included in a subsetting method. The following table describes the columns in the V1REP03 table:

Column | Type | Length | Decimal | Domain
MACHINE_ID | CHARACTER | 10 | 0 | Machine ID
COMPANY | CHARACTER | 10 | 0 | Company
APPLICATION_ID | CHARACTER | 10 | 0 | Application Id
FILE_TYPE | CHARACTER | 4 | 0 | File Type
FILE_OR_TABLE_QUALIFIER | CHARACTER | 128 | 0 | File Or Table Qualifier
FILE_OR_TABLE_NAME | CHARACTER | 128 | 0 | File Or Table Name
RECORD_FORMAT | CHARACTER | 10 | 0 | Record_Format
GDG_VERSION | BINARY | 4 | 0 | GDG Version
Parent topic: Table and Column Descriptions


V1REP04

The V1REP04 table is used as a report that shows all tables included in a subsetting method and where a masking rule is active. The following table describes the columns in the V1REP04 table:

Column | Type | Length | Decimal | Domain
MACHINE_ID | CHARACTER | 10 | 0 | Machine ID
COMPANY | CHARACTER | 10 | 0 | Company
APPLICATION_ID | CHARACTER | 10 | 0 | Application ID
FILE_TYPE | CHARACTER | 4 | 0 | File Type
FILE_OR_TABLE_QUALIFIER | CHARACTER | 128 | 0 | File or Table Qualifier
FILE_OR_TABLE_NAME | CHARACTER | 128 | 0 | File or Table Name
RECORD_FORMAT | CHARACTER | 10 | 0 | Record Format
GDG_VERSION | BINARY | 4 | 0 | GDG Version
Parent topic: Table and Column Descriptions


V1REP05

The V1REP05 table is used as a report that shows all tables that are masked but not included in a subsetting method. The following table describes the columns in the V1REP05 table:

Column | Type | Length | Decimal | Domain
MACHINE_ID | CHARACTER | 10 | 0 | Machine ID
COMPANY | CHARACTER | 10 | 0 | Company
APPLICATION_ID | CHARACTER | 10 | 0 | Application ID
FILE_TYPE | CHARACTER | 4 | 0 | File Type
FILE_OR_TABLE_QUALIFIER | CHARACTER | 128 | 0 | File or Table Qualifier
FILE_OR_TABLE_NAME | CHARACTER | 128 | 0 | File or Table Name
RECORD_FORMAT | CHARACTER | 10 | 0 | Record Format
GDG_VERSION | BINARY | 4 | 0 | GDG Version
Parent topic: Table and Column Descriptions


V1REP06

The V1REP06 table is used as a report that shows all table structures including classes and masking routines. The following table describes the columns in the V1REP06 table:

Column | Type | Length | Decimal | Domain
MACHINE_ID | CHARACTER | 10 | 0 | Machine ID
COMPANY | CHARACTER | 10 | 0 | Company
APPLICATION_ID | CHARACTER | 10 | 0 | Application ID
FILE_TYPE | CHARACTER | 4 | 0 | File Type
FILE_OR_TABLE_QUALIFIER | CHARACTER | 128 | 0 | File or Table Qualifier
FILE_OR_TABLE_NAME | CHARACTER | 128 | 0 | File or Table Name
RECORD_FORMAT | CHARACTER | 10 | 0 | Record Format
GDG_VERSION | BINARY | 4 | 0 | GDG Version
COLUMN_PROGRESSIVE | INTEGER | 4 | 0 | Column Progressive
COLUMN_NAME | CHARACTER | 128 | 0 | Column Name
COLUMN_LENGHT | BINARY | 9 | 0 | Column Length
COLUMN_INTEGER_POSITIONS | BINARY | 4 | 0 | Column Integer Positions
COLUMN_DECIMAL_POSITIONS | BINARY | 4 | 0 | Column Decimal Positions
COLUMN_SIGN | CHARACTER | 1 | 0 | Column Sign
COLUMN_TYPE | CHARACTER | 1 | 0 | Column Type
COLUMN_NULL_CAPABLE | CHARACTER | 1 | 0 | Column Null Capable
COLUMN_VARCHAR | CHARACTER | 1 | 0 | Column Varchar
CLASS_DESCRIPTION | VARCHAR | 30 | 0 | Class Description
ROUTINE_NAME | VARCHAR | 8 | 0 | Routine Name
Parent topic: Table and Column Descriptions


V1REP07

The V1REP07 table is used as a report that shows subsetting logic for all tables. The following table describes the columns in the V1REP07 table:

Column | Type | Length | Decimal | Domain
METHOD | CHARACTER | 10 | 0 | Method
STEP | BINARY | 4 | 0 | Step
MACHINE_ID | CHARACTER | 10 | 0 | Machine ID
COMPANY | CHARACTER | 10 | 0 | Company
APPLICATION_ID | CHARACTER | 10 | 0 | Application ID
FILE_TYPE | CHARACTER | 4 | 0 | File Type
FILE_OR_TABLE_QUALIFIER | CHARACTER | 128 | 0 | File Or Table Qualifier
FILE_OR_TABLE_NAME | CHARACTER | 128 | 0 | File Or Table Name
RECORD_FORMAT | CHAR | 10 | 0 | Record Format
GDG_VERSION | BINARY | 4 | 0 | GDG Version
FILTER_TYPE | VARCHAR | 1 | 0 | Filter Type1
FILTER_SUBTYPE | BINARY | 9 | 0 | Filter Subtype2
1 Filter type values can be:
0 | No filter
1 | Predefined individual filter
2 | Exit routine
3 | Filtered list
4 | Filtered list + exit routine
5 | Predefined individual filter + exit routine
2 Filter subtype values can be:
0 | N/A
1 | Range
2 | List of values
Parent topic: Table and Column Descriptions


V1REP08

The V1REP08 table is used as a report that shows subsetting logic for all tables. The following table describes the columns in the V1REP08 table:

Column | Type | Length | Decimal | Domain
METHOD | CHAR | 10 | 0 | Method
MACHINE_ID | CHAR | 10 | 0 | Machine ID
COMPANY | CHAR | 10 | 0 | Company
CLASS_DESCRIPTION | CHAR | 30 | 0 | Class Description
ROUTINE_NAME | CHAR | 8 | 0 | Routine Name
Parent topic: Table and Column Descriptions


V1REP09

The V1REP09 table is used as a report that shows all tables with their method and step. The following table describes the columns in the V1REP09 table:

Column | Type | Length | Decimal | Domain
METHOD | CHAR | 10 | 0 | Method
STEP | BINARY | 4 | 0 | Step
MACHINE_ID | CHAR | 10 | 0 | Machine ID
COMPANY | CHAR | 10 | 0 | Company
APPLICATION_ID | CHAR | 10 | 0 | Application ID
FILE_TYPE | CHAR | 4 | 0 | File Type
FILE_OR_TABLE_QUALIFIER | CHAR | 128 | 0 | File or Table Qualifier
FILE_OR_TABLE_NAME | CHAR | 128 | 0 | File or Table Name
RECORD_FORMAT | CHAR | 10 | 0 | Record Format
GDG_VERSION | BINARY | 4 | 0 | GDG Version
OUTPUT_DATA_SET_TYPE | CHARACTER | 4 | 0 | Output Data Set Type
OUTPUT_DATA_SET_NAME | CHARACTER | 128 | 0 | Output Data Set Name
OUTPUT_DATA_SET_VERSION | BINARY | 4 | 0 | Output Data Set Version
Parent topic: Table and Column Descriptions


V1REP10

The V1REP10 table is used as a report that shows all logical relationships between tables, used in subsetting. The following table describes the columns in the V1REP10 table:

Column | Type | Length | Decimal | Domain
METHOD | CHAR | 10 | 0 | Method
MACHINE_ID | CHAR | 10 | 0 | Machine ID
COMPANY | CHAR | 10 | 0 | Company
CLASS_DESCRIPTION | CHAR | 30 | 0 | Class Description
APPLICATION_ID_O | CHAR | 10 | 0 | Application ID Output
FILE_TYPE_O | CHAR | 4 | 0 | File Type Output
FILE_OR_TABLE_QUALIFIER_O | CHAR | 128 | 0 | File or Table Qualifier Output
FILE_OR_TABLE_NAME_O | CHAR | 128 | 0 | File or Table Name Output
RECORD_FORMAT_O | CHAR | 10 | 0 | Record Format Output
GDG_VERSION_O | BINARY | 4 | 0 | GDG Version Output
COLUMN_NAME_O | CHAR | 128 | 0 | Column Name Output
APPLICATION_ID_I | CHAR | 10 | 0 | Application ID Input
FILE_TYPE_I | CHARACTER | 4 | 0 | File Type, Input
FILE_OR_TABLE_QUALIFIER_I | CHARACTER | 128 | 0 | File or Table Qualifier Input
FILE_OR_TABLE_NAME_I | CHARACTER | 128 | 0 | File or Table Name Input
RECORD_FORMAT_I | CHARACTER | 10 | 0 | Record Format Output
GDG_VERSION_I | BINARY | 4 | 0 | GDG Version Input
COLUMN_NAME_I | CHARACTER | 128 | 0 | Column Name Input
Parent topic: Table and Column Descriptions


V1REP11

The V1REP11 table is used as a report that shows all logical relationships between tables, used in subsetting. The following table describes the columns in the V1REP11 table:

Column | Type | Length | Decimal | Domain
METHOD | CHARACTER | 10 | 0 | Method
MACHINE_ID | CHARACTER | 10 | 0 | Machine ID
COMPANY | CHARACTER | 10 | 0 | Company
EXECUTION_DATE | CHARACTER | 8 | 0 | Execution Date
Parent topic: Table and Column Descriptions


V1REP12

The V1REP12 table is used as a report that shows method time statistics. The following table describes the columns in the V1REP12 table:

Column | Type | Length | Decimal | Domain
METHOD | CHARACTER | 10 | 0 | Method
MACHINE_ID | CHARACTER | 10 | 0 | Machine ID
COMPANY | CHARACTER | 10 | 0 | Company
CPU_TIME | DECIMAL | 30 | 0 | CPU Time
ELAPSED_TIME | DECIMAL | 30 | 0 | Elapsed Time
Parent topic: Table and Column Descriptions


V1REP13

The V1REP13 table is used as a report that shows all tables included in a subsetting method and where a reduction rule is active. The following table describes the columns in the V1REP13 table:

Column | Type | Length | Decimal | Domain
MACHINE_ID | CHARACTER | 10 | 0 | Machine ID
COMPANY | CHARACTER | 10 | 0 | Company
APPLICATION_ID | CHARACTER | 10 | 0 | Application ID
FILE_TYPE | CHARACTER | 4 | 0 | File Type
FILE_OR_TABLE_QUALIFIER | CHARACTER | 128 | 0 | File or Table Qualifier
FILE_OR_TABLE_NAME | CHARACTER | 128 | 0 | File or Table Name
RECORD_FORMAT | CHARACTER | 10 | 0 | Record Format
GDG_VERSION | BINARY | 4 | 0 | GDG Version
Parent topic: Table and Column Descriptions


V1REP14

The V1REP14 table is used as a report that shows all parameters of filter by range used in subsetting. The following table describes the columns in the V1REP14 table:

Column | Type | Length | Decimal | Domain
METHOD | CHARACTER | 10 | 0 | Method
STEP | BINARY | 4 | 0 | Step
MACHINE_ID | CHARACTER | 10 | 0 | Machine ID
COMPANY | CHARACTER | 10 | 0 | Company
APPLICATION_ID | CHARACTER | 10 | 0 | Application Id
FILE_TYPE | CHARACTER | 4 | 0 | File Type
FILE_OR_TABLE_QUALIFIER | CHARACTER | 128 | 0 | File Or Table Qualifier
FILE_OR_TABLE_NAME | CHARACTER | 128 | 0 | File Or Table Name
RECORD_FORMAT | CHAR | 10 | 0 | Record Format
INCLUDED_OR_EXCLUDED | BINARY | 4 | 0 | GDG Version
FROM | CHAR | 1 | 0 | Included or Excluded
FROM_VALUE | CHAR | 254 | 0 | From Value
TO_VALUE | CHAR | 254 | 0 | To Value
Parent topic: Table and Column Descriptions


V1REP15

The V1REP15 table is used as a report that shows all parameters of filter by value list used in subsetting. The following table describes the columns in the V1REP15 table:

Column | Type | Length | Decimal | Domain
METHOD | CHARACTER | 10 | 0 | Method
STEP | BINARY | 4 | 0 | Step
MACHINE_ID | CHARACTER | 10 | 0 | Machine ID
COMPANY | CHARACTER | 10 | 0 | Company
APPLICATION_ID | CHARACTER | 10 | 0 | Application Id
FILE_TYPE | CHARACTER | 4 | 0 | File Type
FILE_OR_TABLE_QUALIFIER | CHARACTER | 128 | 0 | File Or Table Qualifier
FILE_OR_TABLE_NAME | CHARACTER | 128 | 0 | File Or Table Name
RECORD_FORMAT | CHAR | 10 | 0 | Record Format
INCLUDED_OR_EXCLUDED | BINARY | 4 | 0 | GDG Version
FROM | CHAR | 1 | 0 | Included or Excluded
VALUE | CHAR | 254 | 0 | Value
Parent topic: Table and Column Descriptions


V1URDCPY

The V1URDCPY table is used for HSURDCPY and HSURDCOM views. The following table describes the columns in the V1URDCPY table:

Column | Type | Length | Decimal | Domain
MACHINEID | CHARACTER | 10 | 0 | machine identifier
CDSOC | CHARACTER | 10 | 0 | company name
MCRECID | BINARY | 4 | 0 | machine/company ID
CPYLIB | CHARACTER | 44 | 0 | copybook library
CPYNAME | CHARACTER | 10 | 0 | copybook name
IOAREANAME | CHARACTER | 30 | 0 | I/O area name
IOAREAPRG | BINARY | 3 | 0 | I/O area progressive
CPYTXT | CHARACTER | 50 | 0 | copybook rext
FLANAL | CHARACTER | 1 | 0 | analysis flag
FLBAD | CHARACTER | 1 | 0 | bad copybook flag
FLBADTYPE | CHARACTER | 1 | 0 | type of anomaly
LANGUAGE | CHARACTER | 3 | 0 | language
FLMORE01 | CHARACTER | 1 | 0 | flag more than one 01 level
FLMOREPRG | CHARACTER | 1 | 0 | flag more than one progressive
ASSOCIATED | CHARACTER | 1 | 0 | I/O area associated
RECLEN | BINARY | 5 | 0 | record length
NUMFLD | BINARY | 5 | 0 | number of fields
NUMFLDPCK | BINARY | 5 | 0 | number of packed fields
NUMFLDBIN | BINARY | 5 | 0 | number of binary fields
Parent topic: Table and Column Descriptions


V1URDFIL

The V1URDFIL table is used for HSURDFIL and HSURDCOM views. The following table describes the columns in the V1URDFIL table:

Column | Type | Length | Decimal | Domain
MACHINEID | CHARACTER | 10 | 0 | machine identifier
CDSOC | CHARACTER | 10 | 0 | company name
MCRECID | BINARY | 4 | 0 | machine/company ID
FILLIB | CHARACTER | 128 | 0 | file library
FILTYPE | CHARACTER | 4 | 0 | file type
TYPNAME | CHARACTER | 128 | 0 | ODBC Database Type Name
FILNAME | CHARACTER | 128 | 0 | file name
FILVER | BINARY | 3 | 0 | file version
RECFMT | CHARACTER | 10 | 0 | format of record
FILRECID | BINARY | 9 | 0 | file ID
TSNAME | CHARACTER | 24 | 0 | tablespace name
DBNAME | CHARACTER | 24 | 0 | database name
OBID | BINARY | 4 | 0 | OBID
APPLID | CHARACTER | 10 | 0 | application ID
FILTXT | CHARACTER | 50 | 0 | text
FREQLOG | CHARACTER | 1 | 0 | logging frequency
FREQSPE | CHARACTER | 1 | 0 | sampling frequency
FREQCORR | CHARACTER | 1 | 0 | data corruption frequency
NUMREC | BINARY | 9 | 0 | number of records
PROCID | CHARACTER | 5 | 0 | process ID
UNLFILTYP | CHARACTER | 4 | 0 | unload file type
UNLFILNAM | CHARACTER | 44 | 0 | unload file name
UNLFILVER | BINARY | 3 | 0 | unload file version
RECLEN | BINARY | 5 | 0 | length of record
FMTSEL | CHARACTER | 8 | 0 | format selector
DATEANA | CHARACTER | 8 | 0 | date of analysis
DATECAL | CHARACTER | 8 | 0 | date of last calculation
DATELOG | CHARACTER | 8 | 0 | date of last log
DATESPE | CHARACTER | 8 | 0 | date of last spectrum
DATECORR | CHARACTER | 8 | 0 | date of last data corruption
MBFTYP | CHARACTER | 1 | 0 | type of file
ANALYSIS | CHARACTER | 1 | 0 | analysis state
RECANAL | BINARY | 9 | 0 | number of record analyzed
CPYLIB | CHARACTER | 44 | 0 | associated copybook PDS
CPYNAME | CHARACTER | 10 | 0 | associated copybook name
IOAREANAME | CHARACTER | 30 | 0 | I/O area name
IOAREAPRG | BINARY | 3 | 0 | I/O area progressive
INFOPROV | CHARACTER | 3 | 0 | information provider
FILNICK | CHARACTER | 10 | 0 | nickname of file
LOGMON | CHARACTER | 1 | 0 | logging enabled
DCMON | CHARACTER | 1 | 0 | data corruption enabled
LOGALARM | CHARACTER | 1 | 0 | logging alarm
DCALARM | CHARACTER | 1 | 0 | data corruption alarm
FLANAG | CHARACTER | 1 | 0 | is demographic
COENAB | CHARACTER | 1 | 0 | data consistency
DATECO | CHARACTER | 8 | 0 | consistency date
FANAGPRO | CHARACTER | 3 | 0 | demographic provider
FILCAT | CHARACTER | 10 | 0 | file category
FLAG01 | CHARACTER | 1 | 0 | Flag 1
FLAG02 | CHARACTER | 1 | 0 | Flag 2
FLAG03 | CHARACTER | 1 | 0 | Flag 3
FLAG04 | CHARACTER | 1 | 0 | Flag 4
FLAG05 | CHARACTER | 1 | 0 | Flag 5
Parent topic: Table and Column Descriptions


V1URDFLF

The V1URDFLF table is used for ANURDFLF and HSURDFIL views. The following table describes the columns in the V1URDFLF table:

Column | Type | Length | Decimal | Domain
FILLIB | CHARACTER | 128 | 0 | file library
FILTYPE | CHARACTER | 4 | 0 | file type
FILNAME | CHARACTER | 128 | 0 | File name
FILVER | BINARY | 3 | 0 | file version
RECFMT | CHARACTER | 10 | 0 | format of record
MCRECID | BINARY | 4 | 0 | machine/company ID
FILRECID | BINARY | 9 | 0 | file ID
FLDRECID | BINARY | 9 | 0 | field ID
FLDDISPL | BINARY | 5 | 0 | field displacement
FLDNAME | CHARACTER | 128 | 0 | field name
FLDLEN | BINARY | 5 | 0 | field length
FLDINT | BINARY | 3 | 0 | field integer
FLDDEC | BINARY | 3 | 0 | field decimal
FLDSIGN | CHARACTER | 1 | 0 | field sign
FLDTYPE | CHARACTER | 1 | 0 | field type
FLDTNUM | DECIMAL | 5 | 0 | ODBC Data Type Number
FLDTNAME | CHARACTER | 128 | 0 | ODBC Data Type Name
FLDNAT | CHARACTER | 1 | 0 | field nature
FLJUSTIFY | CHARACTER | 1 | 0 | justify flag
CLATYPE | BINARY | 3 | 0 | field class
INFOPROV | CHARACTER | 3 | 0 | information origin
MINVAL | CHARACTER | 33 | 0 | minimum value
MAXVAL | CHARACTER | 33 | 0 | maximum value
AVGVAL | DECIMAL | 18 | 0 | average value
SPEDIFVAL | BINARY | 5 | 0 | number of different values
USERPGM | CHARACTER | 10 | 0 | user program
HASSPE | CHARACTER | 1 | 0 | spectrum presence flag
FLSPE | CHARACTER | 1 | 0 | spectrum calculation flag
LOGMON | CHARACTER | 1 | 0 | logging enabled
DCMON | CHARACTER | 1 | 0 | data corruption enabled
LOGALARM | CHARACTER | 1 | 0 | logging alarm
DCALARM | CHARACTER | 1 | 0 | data corruption alarm
NULLCAP | CHARACTER | 1 | 0 | null capable field
VARLEN | CHARACTER | 1 | 0 | field with variable length
ISANAG | CHARACTER | 1 | 0 | demographic field
ESTCARD | BINARY | 9 | 0 | estimated cardinality
ESTCLA | BINARY | 3 | 0 | estimated class
SAMPVAL | CHARACTER | 33 | 0 | sample value
SAMPNBR | BINARY | 9 | 0 | sample OCCURS
FLDTXT | CHARACTER | 50 | 0 | text
ISKEY | CHARACTER | 1 | 0 | primary key
FLCLAVAL | CHARACTER | 1 | 0 | use class value
ANAGPRO | CHARACTER | 3 | 0 | demographic provider
Parent topic: Table and Column Descriptions


V1URDIMC

The V1URDIMC table is used for ANURDIMC and HSURDCOM views. The following table describes the columns in the V1URDIMC table:

Column | Type | Length | Decimal | Domain
MCRECID | BINARY | 4 | 0 | machine/company ID
MACHINEID | CHARACTER | 10 | 0 | machine identifier
CDSOC | CHARACTER | 10 | 0 | company name
CPYLIB | CHARACTER | 44 | 0 | copybook PDS
CPYNAME | CHARACTER | 10 | 0 | copybook name
IOAREANAME | CHARACTER | 30 | 0 | I/O area name
IOAREAPRG | BINARY | 3 | 0 | I/O area progressive
IMAGEPRG | BINARY | 4 | 0 | image number
IMAGE | VARCHAR | 3900 | 0 | image
Parent topic: Table and Column Descriptions


V1URDIMG

The V1URDIMG table is used for ANURDIMG and HSURDFIL views. The following table describes the columns in the V1URDIMG table:

Column | Type | Length | Decimal | Domain
MCRECID | BINARY | 4 | 0 | machine/company ID
FILRECID | BINARY | 9 | 0 | file ID
FILLIB | CHARACTER | 128 | 0 | file library
FILTYPE | CHARACTER | 4 | 0 | file type
FILNAME | CHARACTER | 128 | 0 | file name
FILVER | BINARY | 3 | 0 | file version
RECFMT | CHARACTER | 10 | 0 | format of record
IMAGEPRG | BINARY | 4 | 0 | image number
IMAGE | VARCHAR | 4000 | 0 | image
Parent topic: Table and Column Descriptions


V1URDLOG

The V1URDLOG table is used for ANURDLOG and HSURDFIL views. The following table describes the columns in the V1URDLOG table:

Column | Type | Length | Decimal | Domain
MCRECID | BINARY | 4 | 0 | machine/company ID
FILRECID | BINARY | 9 | 0 | file ID
FILLIB | CHARACTER | 128 | 0 | file library
FILTYPE | CHARACTER | 4 | 0 | file type
FILNAME | CHARACTER | 128 | 0 | file name
FILVER | BINARY | 3 | 0 | file version
RECFMT | CHARACTER | 10 | 0 | format of record
MACHINEID | CHARACTER | 10 | 0 | machine identifier
CDSOC | CHARACTER | 10 | 0 | company name
DTCALC | CHARACTER | 8 | 0 | last tecalculation date
FLDNAME | CHARACTER | 128 | 0 | field name
FLDDISPL | BINARY | 5 | 0 | field position
FLDVAL | CHARACTER | 32 | 0 | field value
DATEFMT | CHARACTER | 10 | 0 | date format
MSGMNE | CHARACTER | 5 | 0 | message type
MSGID | CHARACTER | 7 | 0 | message code
MSGDESC | CHARACTER | 132 | 0 | message description
OCCURS | CHARACTER | 9 | 0 | OCCURS
GRAVITY | CHARACTER | 2 | 0 | severity
FLSTOP | CHARACTER | 1 | 0 | stop elaboration flag
TRCIMG | CHARACTER | 512 | 0 | record image
TRCRRN | BINARY | 9 | 0 | internal RRN
IMPDTTM | TIMESTAMP | 26 | 0 | logging date/time
Parent topic: Table and Column Descriptions


V1URDRFC

The V1URDRFC table is used for ANURDRFC and HSURDFIL views. The following table describes the columns in the V1URDRFC table:

Column | Type | Length | Decimal | Domain
MCRECID | BINARY | 4 | 0 | machine/company ID
FILRECID | BINARY | 9 | 0 | file ID
CPYMCRECID | BINARY | 4 | 0 | copybook machine/company ID
CPYLIB | CHARACTER | 44 | 0 | copybook PDS
CPYNAME | CHARACTER | 10 | 0 | copybook name
IOAREANAME | CHARACTER | 30 | 0 | I/O area name
IOAREAPRG | BINARY | 3 | 0 | I/O area progressive
DELETED | CHARACTER | 1 | 0 | deleted flag
RECLEN | BINARY | 5 | 0 | record length
RECMATCH | BINARY | 5 | 0 | number bytes matching
INFOPROV | CHARACTER | 3 | 0 | information provider
ASSOCIATED | CHARACTER | 1 | 0 | copybook associated
FILLIB | CHARACTER | 128 | 0 | file library
FILTYPE | CHARACTER | 4 | 0 | file type
FILNAME | CHARACTER | 128 | 0 | file name
FILVER | BINARY | 3 | 0 | file version
RECFMT | CHARACTER | 10 | 0 | format of record
Parent topic: Table and Column Descriptions


V2URDFLF

The V2URDFLF table is used for ANURDFLF and HSURDFIL views. The following table describes the columns in the V2URDFLF table:

Column | Type | Length | Decimal | Domain
FILLIB | CHARACTER | 128 | 0 | file library
FILTYPE | CHARACTER | 4 | 0 | file type
FILNAME | CHARACTER | 128 | 0 | file name
FILVER | BINARY | 3 | 0 | file version
RECFMT | CHARACTER | 10 | 0 | format of record
TSNAME | CHARACTER | 24 | 0 | tablespace name
DBNAME | CHARACTER | 24 | 0 | database name
OBID | BINARY | 4 | 0 | OBID
APPLID | CHARACTER | 10 | 0 | application ID
FILTXT | CHARACTER | 50 | 0 | text
FREQLOG | CHARACTER | 1 | 0 | logging frequency
FREQSPE | CHARACTER | 1 | 0 | sampling frequency
FREQCORR | CHARACTER | 1 | 0 | data corruption frequency
NUMREC | BINARY | 9 | 0 | number of records
PROCID | CHARACTER | 5 | 0 | process ID
UNLFILTYP | CHARACTER | 4 | 0 | unload file type
UNLFILNAM | CHARACTER | 44 | 0 | unload file name
UNLFILVER | BINARY | 3 | 0 | unload file version
RECLEN | BINARY | 5 | 0 | length of record
FMTSEL | CHARACTER | 8 | 0 | format selector
DATEANA | CHARACTER | 8 | 0 | date of analysis
DATECAL | CHARACTER | 8 | 0 | date of last calculation
DATELOG | CHARACTER | 8 | 0 | date of last log
DATESPE | CHARACTER | 8 | 0 | date of last spectrum
DATECORR | CHARACTER | 8 | 0 | date of last data corruption
MBFTYP | CHARACTER | 1 | 0 | type of file
ANALYSIS | CHARACTER | 1 | 0 | analysis state
RECANAL | BINARY | 9 | 0 | number of record analyzed
CPYLIB | CHARACTER | 44 | 0 | associated copybook PDS
CPYNAME | CHARACTER | 10 | 0 | associated copybook name
IOAREANAME | CHARACTER | 30 | 0 | I/O area name
IOAREAPRG | BINARY | 3 | 0 | I/O area progressive
INFOPROV | CHARACTER | 3 | 0 | information provider
FILNICK | CHARACTER | 10 | 0 | nickname of file
FLANAG | CHARACTER | 1 | 0 | is demograhpic
COENAB | CHARACTER | 1 | 0 | data consistency
DATECO | CHARACTER | 8 | 0 | consistency date
FANAGPRO | CHARACTER | 3 | 0 | demographic provider
FILCAT | CHARACTER | 10 | 0 | file category
FLAG01 | CHARACTER | 1 | 0 | Flag 1
F LAG02 | CHARACTER | 1 | 0 | Flag 2
F LAG03 | CHARACTER | 1 | 0 | Flag 3
F LAG04 | CHARACTER | 1 | 0 | Flag 4
F LAG05 | CHARACTER | 1 | 0 | Flag 5
MCRECID | BINARY | 4 | 0 | machine/company ID
FILRECID | BINARY | 9 | 0 | file ID
FLDRECID | BINARY | 9 | 0 | field ID
FLDDISPL | BINARY | 5 | 0 | field displacement
FLDNAME | CHARACTER | 128 | 0 | field name
FLDLEN | BINARY | 5 | 0 | field length
FLDINT | BINARY | 3 | 0 | field integer
FLDDEC | BINARY | 3 | 0 | field decimal
FLDSIGN | CHARACTER | 1 | 0 | field sign
FLDTYPE | CHARACTER | 1 | 0 | field type
FLDTNUM | DECIMAL | 5 | 0 | ODBC Data Type Number
FLDTNAME | CHARACTER | 128 | 0 | ODBC Data Type Name
FLDNAT | CHARACTER | 1 | 0 | field nature
FLJUSTIFY | CHARACTER | 1 | 0 | justify flag
CLATYPE | BINARY | 3 | 0 | field class
MINVAL | CHARACTER | 33 | 0 | minimum value
MAXVAL | CHARACTER | 33 | 0 | maximum value
AVGVAL | DECIMAL | 18 | 0 | average value
SPEDIFVAL | BINARY | 5 | 0 | number of different values
USERPGM | CHARACTER | 10 | 0 | user program
HASSPE | CHARACTER | 1 | 0 | spectrum presence flag
FLSPE | CHARACTER | 1 | 0 | spectrum calculation flag
LOGMON | CHARACTER | 1 | 0 | logging enabled for field
DCMON | CHARACTER | 1 | 0 | data corruption enabled
LOGALARM | CHARACTER | 1 | 0 | logging alarm for field
DCALARM | CHARACTER | 1 | 0 | data corruption alarm
NULLCAP | CHARACTER | 1 | 0 | null capable field
VARLEN | CHARACTER | 1 | 0 | field with variable length
ISANAG | CHARACTER | 1 | 0 | demographci field
ESTCARD | BINARY | 9 | 0 | estimated cardinality
ESTCLA | BINARY | 3 | 0 | estimated class
SAMPVAL | CHARACTER | 33 | 0 | sample value
SAMPNBR | BINARY | 9 | 0 | sample OCCURS
FLDTXT | CHARACTER | 50 | 0 | text
ISKEY | CHARACTER | 1 | 0 | primary key
FLCLAVAL | CHARACTER | 1 | 0 | use class value
ANAGPRO | CHARACTER | 3 | 0 | demographic provider
Parent topic: Table and Column Descriptions


V3URDFLF

The V3URDFLF table is used for ANURDFLF and HSURDFIL views. The following table describes the columns in the V3URDFLF table:

Column | Type | Length | Decimal | Domain
FILLIB | CHARACTER | 128 | 0 | file library
FILTYPE | CHARACTER | 4 | 0 | file type
FILNAME | CHARACTER | 128 | 0 | file name
FILVER | BINARY | 3 | 0 | file version
RECFMT | CHARACTER | 10 | 0 | format of record
TSNAME | CHARACTER | 24 | 0 | tablespace name
DBNAME | CHARACTER | 24 | 0 | database name
OBID | BINARY | 4 | 0 | OBID
APPLID | CHARACTER | 10 | 0 | application ID
FILTXT | CHARACTER | 50 | 0 | text
FREQLOG | CHARACTER | 1 | 0 | logging frequency
FREQSPE | CHARACTER | 1 | 0 | sampling frequency
FREQCORR | CHARACTER | 1 | 0 | data corruption frequency
NUMREC | BINARY | 9 | 0 | number of records
PROCID | CHARACTER | 5 | 0 | process ID
UNLFILTYP | CHARACTER | 4 | 0 | unload file type
UNLFILNAM | CHARACTER | 44 | 0 | unload file name
UNLFILVER | BINARY | 3 | 0 | unload file version
RECLEN | BINARY | 5 | 0 | length of record
FMTSEL | CHARACTER | 8 | 0 | format selector
DATEANA | CHARACTER | 8 | 0 | date of analysis
DATECAL | CHARACTER | 8 | 0 | date of last calculation
DATELOG | CHARACTER | 8 | 0 | date of last log
DATESPE | CHARACTER | 8 | 0 | date of last spectrum
DATECORR | CHARACTER | 8 | 0 | date of last data corruption
MBFTYP | CHARACTER | 1 | 0 | type of file
ANALYSIS | CHARACTER | 1 | 0 | analysis state
RECANAL | BINARY | 9 | 0 | number of record Analyzed
CPYLIB | CHARACTER | 44 | 0 | associated copybook PDS
CPYNAME | CHARACTER | 10 | 0 | associated copybook name
IOAREANAME | CHARACTER | 30 | 0 | I/O area name
IOAREAPRG | BINARY | 3 | 0 | I/O area progressive
INFOPROV | CHARACTER | 3 | 0 | information provider
FILNICK | CHARACTER | 10 | 0 | nickname of file
LOGMON | CHARACTER | 1 | 0 | logging enabled for file
LOGALARM | CHARACTER | 1 | 0 | logging alarm for file
FLANAG | CHARACTER | 1 | 0 | is demographic
COENAB | CHARACTER | 1 | 0 | data consistency
DATECO | CHARACTER | 8 | 0 | consistency date
FANAGPRO | CHARACTER | 3 | 0 | demographic provider
FILCAT | CHARACTER | 10 | 0 | file category
FLAG01 | CHARACTER | 1 | 0 | Flag 1
F LAG02 | CHARACTER | 1 | 0 | Flag 2
F LAG03 | CHARACTER | 1 | 0 | Flag 3
F LAG04 | CHARACTER | 1 | 0 | Flag 4
F LAG05 | CHARACTER | 1 | 0 | Flag 5
MCRECID | BINARY | 4 | 0 | machine/company ID
FILRECID | BINARY | 9 | 0 | file ID
FLDRECID | BINARY | 9 | 0 | field ID
FLDDISPL | BINARY | 5 | 0 | field displacement
FLDNAME | CHARACTER | 128 | 0 | field name
FLDLEN | BINARY | 5 | 0 | field length
FLDINT | BINARY | 3 | 0 | field integer
FLDDEC | BINARY | 3 | 0 | field decimal
FLDSIGN | CHARACTER | 1 | 0 | field sign
FLDTYPE | CHARACTER | 1 | 0 | field type
FLDNAT | CHARACTER | 1 | 0 | field nature
FLDTNUM | DECIMAL | 5 | 0 | ODBC Data Type Number
FLDTNAME | CHARACTER | 128 | 0 | ODBC Data Type Name
FLJUSTIFY | CHARACTER | 1 | 0 | justify flag
CLATYPE | BINARY | 3 | 0 | field class
MINVAL | CHARACTER | 33 | 0 | minimum value
MAXVAL | CHARACTER | 33 | 0 | maximum value
AVGVAL | DECIMAL | 18 | 0 | average value
SPEDIFVAL | BINARY | 5 | 0 | number of different values
USERPGM | CHARACTER | 10 | 0 | user program
HASSPE | CHARACTER | 1 | 0 | spectrum presence flag
FLSPE | CHARACTER | 1 | 0 | spectrum calculation flag
DCMON | CHARACTER | 1 | 0 | data corruption enabled
DCALARM | CHARACTER | 1 | 0 | data corruption alarm
NULLCAP | CHARACTER | 1 | 0 | null capable field
VARLEN | CHARACTER | 1 | 0 | field with variable length
ISANAG | CHARACTER | 1 | 0 | demographic field
ESTCARD | BINARY | 9 | 0 | estimated cardinality
ESTCLA | BINARY | 3 | 0 | estimated class
SAMPVAL | CHARACTER | 33 | 0 | sample value
SAMPNBR | BINARY | 9 | 0 | sample OCCURS
FLDTXT | CHARACTER | 50 | 0 | text
ISKEY | CHARACTER | 1 | 0 | primary key
FLCLAVAL | CHARACTER | 1 | 0 | use class value
ANAGPRO | CHARACTER | 3 | 0 | demographic provider
Parent topic: Table and Column Descriptions


XPURDFLC

The XPURDFLC table is used for temporary copybook/include fields and is populated during the Analyze Copybook phase. The following table describes the columns in the XPURDFLC table:

Column | Type | Length | Decimal | Domain
MCRECID | BINARY | 4 | 0 | machine/company ID
CPYLIB | CHARACTER | 44 | 0 | copybook library
CPYNAME | CHARACTER | 10 | 0 | copybook name
STMTNBR | BINARY | 7 | 0 | statement number
IOAREANAME | CHARACTER | 30 | 0 | I/O area name
IOAREAPRG | BINARY | 3 | 0 | I/O area progressive
IOAREASCP | BINARY | 3 | 0 | I/O area scope
FLDDISPL | BINARY | 5 | 0 | field displacement
FLDNAME | CHARACTER | 128 | 0 | field name
FLDSCP | BINARY | 3 | 0 | field scope
FLDLEVEL | CHARACTER | 2 | 0 | field level
IOAREANBR | BINARY | 5 | 0 | I/O area number
FLDNBR | BINARY | 5 | 0 | field number
FLDLEN | BINARY | 5 | 0 | field length
FLDINT | BINARY | 3 | 0 | field integer
FLDDEC | BINARY | 3 | 0 | field decimal
FLDSIGN | CHARACTER | 1 | 0 | field sign
FLDTYPE | CHARACTER | 1 | 0 | field type
FLDOCC | BINARY | 5 | 0 | field OCCURS
FLDOCCTOT | BINARY | 5 | 0 | total OCCURS elements
FLDOCCLVL | BINARY | 3 | 0 | OCCURS level
FLDOCCPRG | BINARY | 5 | 0 | OCCURS progressive
FLDOCCSHF | BINARY | 5 | 0 | shift to next element displacement
FLDREDEF | CHARACTER | 1 | 0 | field REDEFINES
FLDREDEFC | CHARACTER | 1 | 0 | field REDEFINES clause
FLDNBRREDE | BINARY | 5 | 0 | number of REDEFINES field
FLDGROUP | CHARACTER | 1 | 0 | Group field
FLJUSTIFY | CHARACTER | 1 | 0 | Justify flag
Parent topic: Table and Column Descriptions


Sequential Files

This chapter lists information for sequential files.

  • Class List Interface
    Describes the class list interface sequential file, which is used to import and export class information.
  • Combined List Interface
    Describes the combined list interface sequential file, which is used to import and export combined information.
  • DB2 Catalog Synchronization Information
    Contains DB2 Catalog Synchronization Information.
  • Load Copy Information from External Interface
    Describes the Load Copy Information from External Interface sequential file.
  • Load Data Store Information from External Interface
    Describes the Load Data Store Information from External Interface.
  • Load Referential Integrity Relation Information
    Describes the Load Referential Integrity Relation Information sequential file.
  • Import Classification from Data Dictionary
    Describes the Import Classification from Data Dictionary sequential file.
  • IMPSRC - Modified Sources
    Describes the IMPSRC sequential file.
  • Method List Interface
    Describes the method list interface sequential file, which is used to import and export method information.
  • Multi-format List Interface
    Describes the multi-format list interface sequential file, which is used to import and export multi-format information.
Parent topic: Data Model Guide


Class List Interface

The class list interface is used to import and export class information, and must be provided as a sequential file defined as FB with record length 162.

The following table describes the columns in the class list interface:

Column | Type | Length | Decimal | Domain
CLATYPE | INTEGER | 4 | 0 | Class number
CLACLS | CHARACTER | 1 | 0 | Charcode
DSCLATYPE | CHARACTER | 30 | 0 | Class description
DRCLATYPE | CHARACTER | 7 | 0 | Class short description
EUROSENS | CHARACTER | 1 | 0 | Hilight flag
FLALFA | CHARACTER | 1 | 0 | Alphanumeric Field
FLZND | CHARACTER | 1 | 0 | Zoned Field
FLPCK | CHARACTER | 1 | 0 | Packed Field
FLBIN | CHARACTER | 1 | 0 | Binary Field
FLEDT | CHARACTER | 1 | 0 | Edited Field
LENMIN | INTEGER | 9 | 0 | Minimum Length
LENMAX | INTEGER | 9 | 0 | Maximum Length
INTMIN | INTEGER | 9 | 0 | Minimum Integer
INTMAX | INTEGER | 9 | 0 | Maximum Integer
DECMIN | INTEGER | 9 | 0 | Minimum Decimal
DECMAX | INTEGER | 9 | 0 | Maximum Decimal
SUPCLASS | CHARACTER | 1 | 0 | Superclass
EDITMSK | CHARACTER | 32 | 0 | Edit Mask
CHGPGM | CHARACTER | 8 | 0 | Masking Program
Parent topic: Sequential Files


Combined List Interface

The combined list interface is used to import and export combined information, and must be provided as a sequential file defined as FB with record length 311.

The following tables describe the columns in the combined list interface:

Table 1. Combined data element - header
Column | Type | Length | Decimal | Domain
RECTYPE | CHARACTER | 1 | 0 | Always set to H
FILTYPE | CHARACTER | 4 | 0 | Data store type
FILNAME | CHARACTER | 128 | 0 | Data store name
FILLIB | CHARACTER | 128 | 0 | Schema name or DBD name
FILVER | INTEGER | 3 | 0 | GDG version (0 for non-GDG)
RECFMT | CHARACTER | 10 | 0 | Record format
FLDNAME | CHARACTER | 30 | 0 | Combined data element name
Table 2. Combined data element - detail
Column | Type | Length | Decimal | Domain
RECTYPE | CHARACTER | 1 | 0 | Always set to D
PRGCBZ | INTEGER | 9 | 0 | Prog.Fields Group
CBZTXT | CHARACTER | 50 | 0 | Fields Group Text
FLDNAME | CHARACTER | 30 | 0 | Data Element Name
FLDRECID | INTEGER | 9 | 0 | Data Element ID
CTRLTAB | CHARACTER | 18 | 0 | Control Table
CTRLPGM | CHARACTER | 10 | 0 | Control Program
EDTSIG | CHARACTER | 1 | 0 | Flag Sign on String
EDTDEC | CHARACTER | 1 | 0 | Flag Decimal on String
Table 3. Combined data element - extension
Column | Type | Length | Decimal | Domain
RECTYPE | CHARACTER | 1 | 0 | Always set to E
PRGCBZ | INTEGER | 9 | 0 | Prog.Fields Group
FLDNAME | CHARACTER | 30 | 0 | Data Element Name
FLDRECID | INTEGER | 9 | 0 | Data Element ID
EXTTYPE | CHARACTER | 1 | 0 | Extended Value Type
EXTPRG | INTEGER | 9 | 0 | Extended Value Progressive
EXTPOS | INTEGER | 9 | 0 | Position for Substring
EXTLEN | INTEGER | 9 | 0 | Length for Substring
EXTVAL | CHARACTER | 100 | 0 | Value for Prefix and Suffix
EXTHEX | CHARACTER | 1 | 0 | Value is Hexadecimal
EXTROUTINE | CHARACTER | 8 | 0 | Routine Name
Parent topic: Sequential Files


DB2 Catalog Synchronization Information

The DB2 Catalog Synchronization Information sequential file is used for the layout of a sequential file in text format and is populated during the execution of the Data Subset Extraction batch process.

  • DB2 Catalog Synchronization - HEADER
    Describes the columns in the DB2 Catalog Synchronization Information sequential file for HEADER
  • DB2 Catalog Synchronization - INTERNAL KEY
    Describes the columns in the DB2 Catalog Synchronization Information sequential file for INTERNAL KEY.
  • DB2 Catalog Synchronization - EXTERNAL KEY
    Describes the columns in the DB2 Catalog Synchronization Information sequential file for EXTERNAL KEY.
  • DB2 Catalog Synchronization - IMAGE COPY
    Describes the columns in the DB2 Catalog Synchronization Information sequential file for IMAGE COPY.
Parent topic: Sequential Files


DB2 Catalog Synchronization - HEADER

The following table describes the columns in the DB2 Catalog Synchronization Information sequential file for HEADER:

Column | Type | Length | Decimal | Domain
RECFMT | CHARACTER | 001-004 | 4 | HEAD
Separator | CHARACTER | 005-005 | 1 | must be blank
TBNAME | CHARACTER | 006-133 | 128 | table name (mandatory field)
Separator | CHARACTER | 134-134 | 1 | must be blank
TBCREATOR | CHARACTER | 135-262 | 128 | table creator ( mandatory field)
Separator | CHARACTER | 263-263 | 1 | must be blank
CARD | CHARACTER | 264-272 | 9 | table cardinality ( mandatory field).
Parent topic: DB2 Catalog Synchronization Information


DB2 Catalog Synchronization - INTERNAL KEY

The following table describes the columns in the DB2 Catalog Synchronization Information sequential file for INTERNAL KEY:

Column | Type | Length | Decimal | Domain
RECFMT | CHARACTER | 001-004 | 4 | IKEY
Separator | CHARACTER | 005-005 | 1 | must be blank
IXNAME | CHARACTER | 006-133 | 128 | index name (mandatory field)
Separator | CHARACTER | 134-134 | 1 | must be blank
IXCREATOR | CHARACTER | 135-262 | 128 | index creator ( mandatory field)
Separator | CHARACTER | 263-263 | 1 | must be blank
COLSEQ | CHARACTER | 264-267 | 4 | column sequence ( mandatory field)
Separator | CHARACTER | 268-268 | 1 | must be blank
COLNAME | CHARACTER | 269-298 | 30 | column name ( mandatory field)
Separator | CHARACTER | 299-299 | 1 | must be blank
COLNO | CHARACTER | 300-303 | 4 | column number ( mandatory field)
Parent topic: DB2 Catalog Synchronization Information


DB2 Catalog Synchronization - EXTERNAL KEY

The following table describes the columns in the DB2 Catalog Synchronization Information sequential file for EXTERNAL KEY:

Column | Type | Length | Decimal | Domain
RECFMT | CHARACTER | 001-004 | 4 | EKEY
Separator | CHARACTER | 005-005 | 1 | must be blank
IXNAME | CHARACTER | 006-133 | 128 | index name (mandatory field)
Separator | CHARACTER | 134-134 | 1 | must be blank
IXCREATOR | CHARACTER | 135-262 | 128 | index creator ( mandatory field)
Separator | CHARACTER | 263-263 | 1 | must be blank
COLSEQ | CHARACTER | 264-267 | 4 | column sequence ( mandatory field)
Separator | CHARACTER | 268-268 | 1 | must be blank
COLNAME | CHARACTER | 269-298 | 30 | column name ( mandatory field).
Separator | CHARACTER | 299-299 | 1 | must be blank
COLNO | CHARACTER | 300-303 | 4 | column number ( mandatory field)
Parent topic: DB2 Catalog Synchronization Information


DB2 Catalog Synchronization - IMAGE COPY

The following table describes the columns in the DB2 Catalog Synchronization Information sequential file for IMAGE COPY:

Column | Type | Length | Decimal | Domain
RECFMT | CHARACTER | 001-004 | 4 | ICPY
Separator | CHARACTER | 005-005 | 1 | must be blank
DBNAME | CHARACTER | 006-029 | 24 | database name (mandatory field)
Separator | CHARACTER | 030-030 | 1 | must be blank
TSNAME | CHARACTER | 031-054 | 24 | tablespace creator (mandatory field)
Separator | CHARACTER | 055-055 | 1 | must be blank
DSNUM | CHARACTER | 056-064 | 9 | data set number (mandatory field)
Separator | CHARACTER | 065-065 | 1 | must be blank
ICDATE | CHARACTER | 066-073 | 8 | image copybook date (mandatory field)
Separator | CHARACTER | 074-074 | 1 | must be blank
DSNAME | CHARACTER | 075-118 | 44 | data set name (mandatory field)
Separator | CHARACTER | 119-119 | 1 | must be blank
DSTIMESTAMP | CHARACTER | 120-145 | 26 | timestamp (mandatory field)
Parent topic: DB2 Catalog Synchronization Information


Load Copy Information from External Interface

The Load Copy Information from External Interface sequential file is used for the layout of a sequential file in text format and is populated during the Handle Copybook phase from a flat file interface. The Sample data folder on the installation CD contains the file seqcpy, which is a sample of this interface.

The following table describes the columns in the Load Copy Information from External Interface sequential file:

Column | Type | Length | Decimal | Domain
CPYLIB | CHARACTER | 001-044 | 44 | copybook library (mandatory field)
Separator | CHARACTER | 045-045 | 1 | must be blank
CPYNAME | CHARACTER | 046-055 | 10 | copybook name (mandatory field). Can be blank or a partial name (with '*' or '?')
Separator | CHARACTER | 056-056 | 1 | must be blank
LOADTYPE | CHARACTER | 057-057 | 1 | load only handled languages flag ( '/' for yes, blank for no)
Separator | CHARACTER | 058-058 | 1 | must be blank
REPLACE | CHARACTER | 059-059 | 1 | flag replace parameter
Separator | CHARACTER | 060-060 | 1 | must be blank
ASSOCIATE | CHARACTER | 061-061 | 1 | flag associate parameter
Separator | CHARACTER | 062-062 | 1 | must be blank
TOLLCPY | INTEGER | 063-064 | 2 | copybook tolerance parameter
Separator | CHARACTER | 065-065 | 1 | must be blank
DECSEP | CHARACTER | 066-066 | 1 | COBOL decimal separator parameter
Separator | CHARACTER | 067-067 | 1 | must be blank
MAXIOAREA | INTEGER | 068-070 | 3 | maximum progressive of I/O area parameter
Separator | CHARACTER | 071-071 | 1 | must be blank
SRCINZ | INTEGER | 072-074 | 3 | begin PL/I source (details below)
Separator | CHARACTER | 075-076 | 1 | must be blank
SRCLEN | INTEGER | 076-078 | 3 | length PL/I source (details below)
  • Flag replace Parameter
    Describes the Flag replace parameter.
  • Flag associate Parameter
    Describes the Flag associate Parameter.
  • Copy Tolerance Parameter
    Describes the Copy Tolerance Parameter.
  • COBOL Decimal Seperator Parameter
    Describes the COBOL Decimal Seperator Parameter.
  • Maximum Progressive of I/O Area Parameter
    Describes the Maximum Progressive of I/O Area Parameter.
  • Statement Length in PL/I Source Parameter
    Describes the Statement Length in PL/I Source Parameter.
Parent topic: Sequential Files


Flag replace Parameter

The Flag replace parameter indicates the conditions under which a copybook must be analyzed:

  • Performs the analysis only if the copybook has not been analyzed.
  • Performs the analysis if the copybook has been analyzed but not associated.
  • Performs the analysis even if the copybook has already been analyzed and associated.
Note: If the default defined machine ID and company level are assumed, leave this parameter blank.
Parent topic: Load Copy Information from External Interface


Flag associate Parameter

The Flag associate parameter lets you choose whether to associate the file directly to the copybook describing its trace. The association algorithm will only be executed on copybooks analyzed until then. Therefore, this option is recommended only when the whole packet of copies belonging to the application has been analyzed. Values allowed:

  • / = yes
  • blank = not
Note: This parameter is intended for future use.
Parent topic: Load Copy Information from External Interface


Copy Tolerance Parameter

The Copy Tolerance parameter lets you specify a percentage of tolerance between the length of the file layout and that of the copybook describing it. If you set this parameter to 0 (zero), these length values must coincide. Setting a tolerance to a value different from zero may be useful when ‘FILLER’ fields involving the use of a copy layout longer than the file layout are used in the copies. It is a 2-digit numeric value, padded with zeroes at the left.

Note: If the default defined machine ID and company level are assumed, leave this parameter blank.
Parent topic: Load Copy Information from External Interface


COBOL Decimal Seperator Parameter

The COBOL Decimal Separator parameter lets you specify the decimal separator within the COBOL copybook (comma or period). Use value 1 for period and 2 for comma.

Note: If the default defined machine ID and company level are assumed, leave this parameter blank.
Parent topic: Load Copy Information from External Interface


Maximum Progressive of I/O Area Parameter

The Maximum Progressive of I/O Area parameter lets you specify the maximum number of incongruent formats an I/O area can describe through the redefines (within COBOL copies) or readdressing (within PL/I includes) clause. It is a 3-digit numeric value padded with zeroes at the left.

Note: If the default defined machine ID and company level are assumed, leave this parameter blank.
Parent topic: Load Copy Information from External Interface


Statement Length in PL/I Source Parameter

The Statement Length in PL/I Source parameter lets you specify the length of the source row within the PL/I includes. They are 3-digit numeric values, padded with zeros at the left.

Note: If the default defined machine ID and company level are assumed, leave this parameter blank.
Parent topic: Load Copy Information from External Interface


Load Data Store Information from External Interface

The Load Data Store Information from External Interface sequential file is used for the layout of a sequential file in text format and is populated when file information is loaded from an external interface.

Note: The sample data folder on the installation CD contains the file extseq2, which is a sample of this interface. In the same folder, the sample file external contains an example of the input to be used for the toolkit in order to generate this interface (for DB2 only).
  • DB2 Databases
    Describes the layout versions of the Load Data Store Information from External Interface sequential file for DB2 databases.
  • VSAM Databases
    Describes columns in the VSAM Databases Load Data Store Information from External Interface sequential file for VSAM databases.
  • Sequential and GDG Databases
    Describes the columns in the Load Data Store Information from External Interface sequential file for sequential and GDG Databases.
  • DL/I Databases
    Describes the columns in the Load Data Store Information from External Interface sequential file for DL/I databases.
  • ADABAS Data Stores
    Describes tables for ADABAS Data Stores.
Parent topic: Sequential Files


DB2 Databases

This section lists the layout versions of the Load Data Store Information from External Interface sequential file for DB2 databases.

  • Direct Access – API Load Sequential File
    Describes the Direct Access – API Load Sequential File for DB2.
  • Direct Access – HEADER
    Describes the Direct Access – HEADER.
  • Direct Access – DETAIL
    Describes the Direct Access – DETAIL for DB2.
  • Access from Unload
    Describes Access from Unload for DB2.
Parent topic: Load Data Store Information from External Interface


Direct Access – API Load Sequential File

The following table describes the columns in the Load Data Store Information from External Interface sequential file for DB2 databases using direct access:

Note: This is used as input for the Load Sequential File with DB2 Catalog job.
Column | Type | Length | Decimal | Domain
FILETYPE | CHARACTER | 0 01 -004 | 4 | DB2D
Separator | CHARACTER | 005-005 | 1 | must be blank
FILNAME | CHARACTER | 006-133 | 128 | table name (mandatory field).
Separator | CHARACTER | 134-134 | 1 | must be blank
FILLIB | CHARACTER | 135- 262 | 128 | table creator (mandatory field).
Separator | CHARACTER | 263-263 | 1 | must be blank
APPLID | CHARACTER | 264-273 | 10 | application code
Parent topic: DB2 Databases


Direct Access – HEADER

The following table describes the columns in the Load Data Store Information from External Interface sequential file for DB2 databases using direct access for HEADER:

Note: This is produced as the byproduct of performing the Load Sequential File with DB2 Catalog job.
Column | Type | Length | Decimal | Domain
FILETYPE | CHARACTER | 001-004 | 4 | DB2D
Separator | CHARACTER | 005-005 | 1 | must be blank
FILNAME | CHARACTER | 006-133 | 128 | table name (mandatory field)
Separator | CHARACTER | 134-134 | 1 | must be blank
FILLIB | CHARACTER | 135-262 | 128 | table creator ( mandatory field)
Separator | CHARACTER | 263-263 | 1 | must be blank
APPLID | CHARACTER | 264-273 | 10 | application code
Separator | CHARACTER | 274-274 | 1 | must be blank
TSNAME | CHARACTER | 275-298 | 24 | tablespace name
Separator | CHARACTER | 299-299 | 1 | must be blank
DBNAME | CHARACTER | 300-323 | 24 | database name ( mandatory field)
Separator | CHARACTER | 324-324 | 1 | must be blank
OBID | INTEGER | 325-328 | 4 | OBID ( mandatory field)
Separator | CHARACTER | 329-329 | 1 | must be blank
FILTXT | CHARACTER | 330-379 | 50 | text
Separator | CHARACTER | 380-380 | 1 | must be blank
NUMREC | INTEGER | 381-389 | 9 | number of records (if not specified, it will be considered as 5000)
Separator | CHARACTER | 390-390 | 1 | must be blank
RECLEN | INTEGER | 391-395 | 5 | length of records (mandatory field)
Parent topic: DB2 Databases


Direct Access – DETAIL

The following table describes the columns in the Load Data Store Information from External Interface sequential file for DB2 databases using direct access for DETAIL:

Note: This is produced as the byproduct of performing the Load Sequential File with DB2 Catalog job.
Column | Type | Length | Decimal | Domain
FILETYPE | CHARACTER | 001-004 | 4 | DB2F
Separator | CHARACTER | 005-005 | 1 | must be blank
FLDDISPL | INTEGER | 006-010 | 5 | field displacement (mandatory field)
Separator | CHARACTER | 011-011 | 1 | must be blank
FLDNAME | CHARACTER | 012-041 | 30 | field name ( mandatory field)
Separator | CHARACTER | 042-042 | 1 | must be blank
FLDLEN | INTEGER | 043-047 | 5 | field length ( mandatory field)
Separator | CHARACTER | 048-048 | 1 | must be blank
FLDINT | INTEGER | 049-051 | 3 | field integer ( mandatory field)
Separator | CHARACTER | 052-052 | 1 | must be blank
FLDDEC | INTEGER | 053-055 | 3 | field decimal ( mandatory field)
Separator | CHARACTER | 056-056 | 1 | must be blank
FLDTYPE | CHARACTER | 057-057 | 1 | field type ( mandatory field)
Separator | CHARACTER | 058-058 | 1 | must be blank
CLATYPE | INTEGER | 059-062 | 4 | field class
Separator | CHARACTER | 063-063 | 1 | must be blank
NULLCAP | CHARACTER | 064-064 | 1 | null capable field ( mandatory field). Blank = N
Separator | CHARACTER | 065-065 | 1 | must be blank
VARLEN | CHARACTER | 066-066 | 1 | field with variable length
Separator | CHARACTER | 067-067 | 1 | must be blank
FLDTXT | CHARACTER | 068-117 | 50 | text
Parent topic: DB2 Databases


Access from Unload

The following table describes the columns in the Load Data Store Information from External Interface sequential file for DB2 databases using direct access:

Column | Type | Length | Decimal | Domain
FILETYPE | CHARACTER | 001-004 | 4 | DB2
Separator | CHARACTER | 005-005 | 1 | must be blank
UNLOAD FILE | CHARACTER | 006-055 | 50 | unload data set name (mandatory field)
Separator | CHARACTER | 056-056 | 1 | must be blank
SYSPUNCH | CHARACTER | 057-110 | 54 | SYSPUNCH or DCLGEN (mandatory field)
Separator | CHARACTER | 111-111 | 1 | must be blank
APPLID | CHARACTER | 112-121 | 10 | application code
Parent topic: DB2 Databases


VSAM Databases

The following table describes the columns in the Load Data Store Information from External Interface sequential file for VSAM databases:

Column | Type | Length | Decimal | Domain
FILETYPE | CHARACTER | 001-004 | 4 | VSAM for non-RRDS VSAM data sets RRDS for RRDS VSAM data sets
Separator | CHARACTER | 005-005 | 1 | must be blank
FILENAME | CHARACTER | 006-049 | 44 | data set name (mandatory field).
Separator | CHARACTER | 050-050 | 1 | must be blank
NICKNAME | CHARACTER | 051-060 | 10 | nickname
Separator | CHARACTER | 061-061 | 1 | must be blank
UNLOAD FILE | CHARACTER | 062-111 | 50 | unload data set name
Separator | CHARACTER | 112-112 | 1 | must be blank
FMTSEL | CHARACTER | 113-120 | 8 | format selector
Separator | CHARACTER | 121-121 | 1 | must be blank
APPLID | CHARACTER | 122-131 | 10 | application code
Separator | CHARACTER | 132-132 | 1 | must be blank
CPYLIB | CHARACTER | 133-176 | 44 | copybook library
Separator | CHARACTER | 177-177 | 1 | must be blank
CPYNAME | CHARACTER | 178-187 | 10 | copybook name
Separator | CHARACTER | 188-188 | 1 | must be blank
IOAREANAME | CHARACTER | 189-218 | 30 | I/O area name
Separator | CHARACTER | 219-219 | 1 | must be blank
IOAREAPRG | INTEGER | 220-222 | 3 | I/O area progressive
Parent topic: Load Data Store Information from External Interface


Sequential and GDG Databases

The following table describes the columns in the Load Data Store Information from External Interface sequential file for sequential and GDG databases:

Column | Type | Length | Decimal | Domain
FILETYPE | CHARACTER | 001-004 | 4 | GDG or SEQ according to file type
Separator | CHARACTER | 005-005 | 1 | must be blank
FILNAME | CHARACTER | 006-055 | 50 | data set name (mandatory field)
Separator | CHARACTER | 056-056 | 1 | must be blank
NICKNAME | CHARACTER | 057-076 | 20 | nickname
Separator | CHARACTER | 077-077 | 1 | must be blank
UNLOAD FILE | CHARACTER | 078-127 | 50 | unload data set name
Separator | CHARACTER | 128-128 | 1 | must be blank
FMTSEL | CHARACTER | 129-136 | 8 | format selector
Separator | CHARACTER | 137-137 | 1 | must be blank
APPLID | CHARACTER | 138-147 | 10 | application code
Separator | CHARACTER | 148-148 | 1 | must be blank
CPYLIB | CHARACTER | 149-192 | 44 | copybook library
Separator | CHARACTER | 193-193 | 1 | must be blank
CPYNAME | CHARACTER | 194-203 | 10 | copybook name
Separator | CHARACTER | 204-204 | 1 | must be blank
IOAREANAME | CHARACTER | 205-234 | 30 | I/O area name
Separator | CHARACTER | 235-235 | 1 | must be blank
IOAREAPRG | INTEGER | 236-238 | 3 | I/O area progressive
Note: You can specify the FILETYPE parameter for GDG files in either of following ways:
  • When using the relative name (signed integer) for FILENAME, the FILETYPE must be GDG. For example:
    GDG  XXXXXX.GDG.TEST(0)
GDG  XXXXXX.GDG.TEST(-1)
GDG  XXXXXX.GDG.TEST(-2)
  • When using the absolute generation name for FILENAME, the FILETYPE can be GDG or SEQ. For example:
    GDG   XXXXXX.GDG.TEST.G0001V00
GDG  XXXXXX.GDG.TEST.G0002V00
GDG  XXXXXX.GDG.TEST.G0003V00

    Or

    SEQ   XXXXXX.GDG.TEST.G0001V00
SEQ  XXXXXX.GDG.TEST.G0002V00
SEQ  XXXXXX.GDG.TEST.G0003V00

For more information on GDG, see GDG Support in your Process for z/OS Guide.

Parent topic: Load Data Store Information from External Interface


DL/I Databases

The following table describes the columns in the Load Data Store Information from External Interface sequential file for DL/I databases.

Note: DL/I multi-format segments are not supported.
Column | Type | Length | Decimal | Domain
FILETYPE | CHARACTER | 001-004 | 4 | DL/I
Separator | CHARACTER | 005-005 | 1 | must be blank
DBDNAME | CHARACTER | 006-013 | 8 | DBD name (mandatory field)
Separator | CHARACTER | 014-014 | 1 | must be blank
UNLOAD FILE | CHARACTER | 015-064 | 50 | unload data set name (mandatory field)
Separator | CHARACTER | 065-065 | 1 | must be blank
PROCID | CHARACTER | 066-070 | 5 | process ID (mandatory field)
Separator | CHARACTER | 071-071 | 1 | must be blank
APPLID | CHARACTER | 072-081 | 10 | application code
Separator | CHARACTER | 082-082 | 1 | must be blank
CPYLIB | CHARACTER | 083-126 | 44 | copybook library
Separator | CHARACTER | 127-127 | 1 | must be blank
CPYNAME | CHARACTER | 128-137 | 10 | copybook name
Separator | CHARACTER | 138-138 | 1 | must be blank
IOAREANAME | CHARACTER | 139-168 | 30 | I/O area name
Separator | CHARACTER | 169-169 | 1 | must be blank
IOAREAPRG | INTEGER | 170-172 | 3 | I/O area progressive
Separator | CHARACTER | 173-173 | 1 | must be blank
SEGMNTNAME | CHARACTER | 174-181 | 8 | Segment Name (mandatory field only for the manual association segment/copybook)
Parent topic: Load Data Store Information from External Interface


ADABAS Data Stores

This section lists the layout versions of the Load Data Store Information from External Interface sequential file for ADABAS data stores.

  • ADABAS – API Load Sequential File - HEADER
    Describes the columns in the Load Data Store Information from External Interface sequential file for ADABAS Data Store for HEADER.
  • ADABAS Data Store – DETAIL
    Describes the columns in the Load Data Store Information from External Interface sequential file for ADABAS Data Store for DETAIL.
Parent topic: Load Data Store Information from External Interface


ADABAS – API Load Sequential File - HEADER

The following table describes the columns in the Load Data Store Information from External Interface sequential file for ADABAS Data Store for HEADER:

Column | Type | Length | Decimal | Domain
FILETYPE | CHARACTER | 001-004 | 4 | ADAD
Separator | CHARACTER | 005-005 | 1 | must be blank
FILNAME | CHARACTER | 006-055 | 50 | database name + file name + number of files (mandatory field)
Separator | CHARACTER | 056-056 | 1 | must be blank
FILLIB | CHARACTER | 057-110 | 54 | database name (mandatory field)
Separator | CHARACTER | 111-111 | 1 | must be blank
APPLID | CHARACTER | 112-121 | 10 | application code
Separator | CHARACTER | 122-122 | 1 | must be blank
TSNAME | CHARACTER | 123-130 | 8 | tablespace name (mandatory field)
Separator | CHARACTER | 131-131 | 1 | must be blank
DBNAME | CHARACTER | 132-149 | 8 | database number (mandatory field)
Separator | CHARACTER | 140-140 | 1 | must be blank
OBID | INTEGER | 141-144 | 4 | OBID (mandatory field)
Separator | CHARACTER | 145-145 | 1 | must be blank
FILTXT | CHARACTER | 146-195 | 50 | text
Separator | CHARACTER | 196-196 | 1 | must be blank
NUMREC | INTEGER | 197-205 | 9 | number of records (if not specified, it will be considered as 5000)
Separator | CHARACTER | 206-206 | 1 | must be blank
RECLEN | INTEGER | 207-211 | 5 | length of records (mandatory field)
Separator | CHARACTER | 212-212 | 1 | must be blank
IOAREA | INTEGER | 213-242 | 30 | I/O area name (mandatory field)
Parent topic: ADABAS Data Stores


ADABAS Data Store – DETAIL

The following table describes the columns in the Load Data Store Information from External Interface sequential file for ADABAS Data Store for DETAIL:

Column | Type | Length | Decimal | Domain
FILETYPE2 | CHARACTER | 001-004 | 4 | ADAF
Separator | CHARACTER | 005-005 | 1 | must be blank
FLDDISPL | INTEGER | 006-010 | 5 | Field displacement (mandatory field)
Separator | CHARACTER | 011-011 | 1 | must be blank
FLDNAME | CHARACTER | 012-041 | 30 | field name (mandatory field)
Separator | CHARACTER | 042-042 | 1 | must be blank
FLDLEN | INTEGER | 043-047 | 5 | field length (mandatory field)
Separator | CHARACTER | 048-048 | 1 | must be blank
FLDINT | INTEGER | 049-051 | 3 | field Integer (mandatory field)
Separator | CHARACTER | 052-052 | 1 | must be blank
FLDDEC | INTEGER | 053-055 | 3 | field Decimal (mandatory field)
Separator | CHARACTER | 056-056 | 1 | must be blank
FLDTYPE | CHARACTER | 057-057 | 1 | field type (mandatory field)
Separator | CHARACTER | 058-058 | 1 | must be blank
CPYLIB | CHARACTER | 059-102 | 44 | tablespace number (mandatory field)
Separator | CHARACTER | 103-103 | 1 | must be blank
CPYNAME | CHARACTER | 104-113 | 10 | database number (mandatory field)
Separator | CHARACTER | 114-114 | 1 | must be blank
IOAREANAME | CHARACTER | 115-144 | 30 | I/O area name (mandatory field)
Separator | CHARACTER | 145-145 | 1 | must be blank
FLDLEVEL | CHARACTER | 146-147 | 2 | level of field (mandatory field)
Separator | CHARACTER | 148-148 | 1 | must be blank
IOAREANBR | INTEGER | 149-153 | 5 | I/O area nunmber (mandatory field)
Separator | CHARACTER | 154-154 | 1 | must be blank
FLDNBR | INTEGER | 155-159 | 5 | field number (mandatory field)
Separator | CHARACTER | 160-160 | 1 | must be blank
FLDSIGN | CHARACTER | 161-161 | 1 | field sign(mandatory field)
Separator | CHARACTER | 162-162 | 1 | must be blank
FLDOCC | INTEGER | 163-167 | 5 | field OCCURS (mandatory field)
Separator | CHARACTER | 168-168 | 1 | must be blank
FLDOCCTOT | INTEGER | 169-173 | 5 | field OCCURS elements (mandatory field)
Separator | CHARACTER | 174-174 | 1 | must be blank
FLDOCCLVL | INTEGER | 175-177 | 3 | OCCURS Level (mandatory field)
Separator | CHARACTER | 178-178 | 1 | must be blank
FLDOCCPRG | INTEGER | 179-183 | 5 | OCCURS progressive (mandatory field)
Separator | CHARACTER | 184-184 | 1 | must be blank
FLDGROUP | CHARACTER | 185-185 | 1 | group field (mandatory field)
Separator | CHARACTER | 186-186 | 1 | must be blank
STMTNBR | INTEGER | 187-193 | 7 | statement number (mandatory field)
Separator | CHARACTER | 194-194 | 1 | must be blank
FLAGFLD | CHARACTER | 195-195 | 1 | flag for TAB. ANURDFLF (mandatory field)
Separator | CHARACTER | 196-196 | 1 | must be blank
FLDRECID | INTEGER | 197-205 | 9 | field ID (mandatory field)
Separator | CHARACTER | 206-206 | 1 | must be blank
FLDNAT | CHARACTER | 207-207 | 1 | field nature (mandatory field)
Parent topic: ADABAS Data Stores


Load Referential Integrity Relation Information

The Load Referential Integrity Relation Information sequential file is used for the layout of a sequential CSV file in text format and is populated during the Data Subsetting phase. The Sample data folder on the installation CD contains the file modref1 which is a sample of this interface.

The following table describes the fields in the Load Referential Integrity Relation Information sequential file:

Field | Type | Maximum Length | Description | Mandatory
Parent_Creator | CHAR | 128 | parent Table Creator | Yes
Parent_Table | CHAR | 128 | parent table name | Yes
Parent_Field | CHAR | 30 | parent column name | Yes
Child_Creator | CHAR | 128 | child table creator | Yes
Child_Table | CHAR | 128 | child table name | Yes
Child_Field | CHAR | 30 | child column name | Yes
File_Type | CHAR | 04 | file type | No
Relation_Name | CHAR | 18 | Name of relation. Suggested if you have to identify different set of fields belonging to the same file, that describe a different relation. | No
Db_name | CHAR | 128 | database name for distributed data | No
Parent topic: Sequential Files


Import Classification from Data Dictionary

The Import Classification from Data Dictionary sequential file is used for the layout of a sequential file in text format and is populated during the Class Assignment phase from the flat file interface. The Sample data folder on the installation CD contains the file modcla which is a sample of this interface.

The following table describes the columns in the Import Classification from Data Dictionary sequential file:

Column | Type | Maximum Length | Domain | Mandatory
FILTYPE | CHARACTER | 4 | file type | Yes (values: DB2, SEQ, DL/I, ADA, GDG, VSAM)
FILLIB | CHARACTER | 128 | DB2 owner | Yes (only for DB2)
FILNAME | CHARACTER | 128 | file name | Yes
FLDNAME | CHARACTER | 30 | field name | Yes
DRCLATYPE | CHARACTER | 7 | class short description | Yes
DSCLATYPE | CHARACTER | 30 | class description | Yes
FILVER | INTEGER | 3 | file version | No (if not present it is assumed to be 0)
RECFMT | CHARACTER | 10 | record format | Yes (only for multiformat files)
EUROSENS | CHARACTER | 1 | highlight flag | No
PRIMCLS | INTEGER | 3 | primary class | No
DATATYP CHARACTER 1 data type:
  • N=Numeric
  • A=Alpha
  • E=Edit
  • B=Binary
  • P=Packed
Yes
LENMIN | INTEGER | 5 | minimum length | No
LENMAX | INTEGER | 5 | maximum length | No
INTMIN | INTEGER | 3 | minimum integer | No
INTMAX | INTEGER | 3 | maximum Integer | No
DECMIN | INTEGER | 3 | minimum decimal | No
DECMAX | INTEGER | 3 | maximum decimal | No
SUPCLASS | CHARACTER | 1 | super class | No
EDITMSK | CHARACTER | 32 | edit mask | No
FLCARD | CHARACTER | 3 | cardinality | No
Parent topic: Sequential Files


IMPSRC - Modified Sources

The IMPSRC sequential file is used for the layout of a sequential file in text format and is populated during the Life Cycle phase (Changed Files Identification (Guided)).

Note: For known restrictions pertaining to the Life Cycle process, see the chapter Life Cycle Introduction in the Micro Focus Data Express – Life Cycle Guide.
  • HEADER
    Describes the columns in the IMPSRC sequential file.
  • DETAIL
    Describes the columns in the IMPSRC sequential file.
Parent topic: Sequential Files


HEADER

The following table describes the columns in the IMPSRC sequential file:

Column | Type | Displacement | Length | Domain
FILETYPE | CHARACTER | 001-004 | 4 | DB2D or ADAD or SEQ
CPYLIB | CHARACTER | 005-132 | 128 | PDS name/owner
CPYNAME | CHARACTER | 133-260 | 128 | copybook name/table name
FLELAB | CHARACTER | 261-261 | 1 | flag file new
TSNAME | CHARACTER | 262-285 | 24 | tablespace name
DBNAME | CHARACTER | 286-309 | 24 | database name
OBID | CHARACTER | 310-313 | 4 | OBID
FILTXT | CHARACTER | 314-363 | 50 | text
NUMREC | INTEGER | 364-372 | 9 | number of records
RECLEN | INTEGER | 373-377 | 5 | length of records
Parent topic: IMPSRC - Modified Sources


DETAIL

The following table describes the columns in the IMPSRC sequential file:

Column | Type | Displacement | Length | Domain
FILETYPE2 | CHARACTER | 001-004 | 4 | DB2T or ADAF
FLDDISPL | INTEGER | 005-009 | 5 | Field Displacement
FLDNAME | CHARACTER | 010-039 | 30 | Field Name
FLDLEN | INTEGER | 040-044 | 5 | Field Length
FLDINT | INTEGER | 045-047 | 3 | Field Integer
FLDDEC | INTEGER | 048-050 | 3 | Field Decimal
FLDTYPE | CHARACTER | 051-051 | 1 | Field Type
CLATYPE | INTEGER | 052-055 | 4 | Field Class
NULLCAP | CHARACTER | 056-056 | 1 | Null Capable
VARLEN | CHARACTER | 057-057 | 1 | Field with Variable Length
FLDTXT | CHARACTER | 058-107 | 50 | Text
FILLER | CHARACTER | 108-177 | 70 | Filler
Parent topic: IMPSRC - Modified Sources


Method List Interface

The method list interface is used to import and export method information, and must be provided as a sequential file defined as FB with record length 606.

The following tables describe the columns in the method list interface:

Table 1. Method - header
Column | Type | Length | Decimal | Domain
RECTYPE | CHARACTER | 1 | 0 | Always set to M
METHOD | CHARACTER | 10 | 0 | Method name
METHODTXT | CHARACTER | 50 | 0 | Method text
DTCALC | CHARACTER | 8 | 0 | Calculation date
GROUPNAME | CHARACTER | 10 | 0 | Group
CREATOR | CHARACTER | 10 | 0 | Creator
ACTIVE | CHARACTER | 1 | 0 | Active flag
COMPLETED | CHARACTER | 1 | 0 | Completed flag
MTHDOC | CHARACTER | 254 | 0 | Method documentation
Table 2. Method - elaboration
Column | Type | Length | Decimal | Domain
RECTYPE | CHARACTER | 1 | 0 | Always set to E
FILTYPE | CHARACTER | 4 | 0 | Data store type
FILNAME | CHARACTER | 128 | 0 | Data store name
FILLIB | CHARACTER | 128 | 0 | Schema name or DBD name
FILVER | INTEGER | 3 | 0 | GDG version (0 for non-GDG)
RECFMT | CHARACTER | 10 | 0 | Record format
STEP | INTEGER | 4 | 0 | Step
OUTCLATYPE | INTEGER | 4 | 0 | Output class id
OUTCLA | CHARACTER | 30 | 0 | Output class
OUTPRG | INTEGER | 4 | 0 | Output progressive
OUTFLDRECID | INTEGER | 9 | 0 | Output field id
OUTFLDNAME | CHARACTER | 30 | 0 | Output field
OUTREC | CHARACTER | 1 | 0 | Website record flag
INCLATYPE | INTEGER | 4 | 0 | Input class id
INCLA | CHARACTER | 30 | 0 | Input class
INPRG | INTEGER | 4 | 0 | Input prg
INFLDRECID | INTEGER | 9 | 0 | Input field id
INFLDNAME | CHARACTER | 30 | 0 | Input field id
FLELAB | CHARACTER | 1 | 0 | Status flag
WRTREC | INTEGER | 9 | 0 | Number of written records
ANDFILT | INTEGER | 4 | 0 | AND filter flag
PROCIDINP | CHARACTER | 5 | 0 | Input process id
UNLINPTYP | CHARACTER | 4 | 0 | Input file type
UNLINPNAM | CHARACTER | 44 | 0 | Input file name
UNLINPVER | INTEGER | 3 | 0 | Input file version (for GDG only)
FMTSEL | CHARACTER | 8 | 0 | Format selector
SLTPGM | CHARACTER | 8 | 0 | Discard program
UNLOUTTYP | CHARACTER | 4 | 0 | Output file type
UNLOUTNAM | CHARACTER | 44 | 0 | Output file name
UNLOUTVER | INTEGER | 3 | 0 | Output file name (for GDG only)
WRTPGM | CHARACTER | 8 | 0 | Write program
Table 3. Method - limit
Column | Type | Length | Decimal | Domain
RECTYPE | CHARACTER | 1 | 0 | Always set to L
WRNLIMIT | INTEGER | 9 | 0 | Limit to number of written records
STRDATE | CHARACTER | 8 | 0 | For future use
ENDDATE | CHARACTER | 8 | 0 | For future use
STRTIME | CHARACTER | 6 | 0 | For future use
ENDTIME | CHARACTER | 6 | 0 | For future use
Table 4. Method - filter
Column | Type | Length | Decimal | Domain
RECTYPE | CHARACTER | 1 | 0 | Always set to F
CLATYPE | CHARACTER | 1 | 0 | Class type
CLASS | INTEGER | 4 | 0 | Class description
FLTPRG | INTEGER | 4 | 0 | Filter progressive
FLTTYPE | CHARACTER | 1 | 0 | Filter type
MACNBR | INTEGER | 4 | 0 | Macro number
ROUTNAME | CHARACTER | 8 | 0 | Routine name
FLTTXT | CHARACTER | 50 | 0 | Filter text
FLTDOC | CHARACTER | 254 | 0 | Filter documentation
Table 5. Method - filter parameter
Column | Type | Length | Decimal | Domain
RECTYPE | CHARACTER | 1 | 0 | Always set to P
CLATYPE | INTEGER | 4 | 0 | Class type
CLASS | INTEGER | 4 | 0 | Class description
FLTPRG | INTEGER | 4 | 0 | Filter progressive
PARMPRG | INTEGER | 4 | 0 | Parameter progressive
VALUE | CHARACTER | 254 | 0 | Parameter value
Parent topic: Sequential Files


Multi-format List Interface

The multi-format list interface is used to import and export multi-format information, and must be provided as a sequential file defined as FB with record length 157.

The following tables describe the columns in the multi-format list interface:

Table 1. Multi-format list for SEQ and GDG
Column | Type | Length | Decimal | Domain | Start offset | End offset
FILTYPE | CHARACTER | 4 |  | GDG or SEQ | 1 | 4
FILNAME | CHARACTER | 50 |  | Data set name | 6 | 55
RECFMT | CHARACTER | 10 |  | Record format | 57 | 66
CPYLIB | CHARACTER | 44 |  | Copybook library | 68 | 111
CPYNAME | CHARACTER | 10 |  | Copybook name | 113 | 122
IOAREANAME | CHARACTER | 30 |  | I/O area name | 124 | 153
IOAREAPRG | CHARACTER | 3 |  | I/O are progressive | 155 | 157
Table 2. Multi-format list for VSAM
Column | Type | Length | Decimal | Domain | Start offset | End offset
FILTYPE | CHARACTER | 4 |  | VSAM | 1 | 4
FILNAME | CHARACTER | 50 |  | Data set name | 6 | 55
RECFMT | CHARACTER | 10 |  | Record format | 57 | 66
CPYLIB | CHARACTER | 44 |  | Copybook library | 68 | 111
CPYNAME | CHARACTER | 10 |  | Copybook name | 113 | 122
IOAREANAME | CHARACTER | 30 |  | I/O area name | 124 | 153
IOAREAPRG | INTEGER | 3 |  | I/O are progressive | 155 | 157
Parent topic: Sequential Files


Data Subset Extraction Guide

Outlines the concepts and procedures used by the Data Subset Extraction module. Data Subset Extraction uses selection criteria specified by the user to create a new test environment from an existing environment.

Data Subset Extraction can be used to:

  • Generate an integral test environment.
  • Generate a reduced test environment.
  • Generate a test environment with data migration.
  • Generate a test environment while applying data masking techniques (for example, to protect sensitive data).

Data Subset Extraction processing comprises the following elements:

  • Methods creation that generates a test environment from the client side.
  • Extraction job submission from the client side.

To process an environment using Data Subset Extraction for the Data Express for z/OS solution, all the files belonging to the environment undergo the following steps:

  1. Loading and analysis of the application copybooks.
  2. Loading and analysis of data stores.
  3. Data store/copybook automatic association.
  4. Definition and association of selection classes with copybooks.
  5. Identification of register table data stores.

For more information, see the Front End Guide.

  • Who should read this guide
  • Data Subset Extraction considerations
  • Getting Started
  • Introduction to Data Subset Extraction window
  • Work with Groups
  • New Method wizard
  • Work with Method
  • Work with Method selection class/filter properties
  • Work with Method work with steps
  • Work with Method method properties
  • Work with Method elaboration properties
  • Work with Method define reload rules
  • Options
  • Data subset extraction jobs
  • Distributed Exporter
    The Distributed Exporter is a utility that creates and provides necessary information to the Extension Technology so that actual subsetting of your distributed data store can occur.
  • Distributed Statistics Loader
  • Work with Method multiple filters
Parent topic: Data Express


Who should read this guide

This guide is for Micro Focus users who are interested in obtaining a subset of data contained in databases that are either developed internally or purchased by external suppliers.

It explains the Data Subset Extraction configuration procedure, and describes the steps required to migrate the files in your application.

The use of Data Express requires a minimum experience of the mainframe platform z/OS.

Before using this product, we recommend that you carefully review the Process Guide for z/OS and Front End Guide.

Parent topic: Data Subset Extraction Guide


Data Subset Extraction considerations

Bear in mind the following points about the Data Subset Extraction module for Data Express:

  • To use Data Express for z/OS, you must have the correct licence. See the Installation Guide for more details.
  • You do not need to analyze the sources of the programs in the application in order to use Data Subset Extraction.
  • The Data Subset Extraction does not perform any operation capable of altering the analyzed data.
  • The Data Subset Extraction module manages all types of data stores processed by the Data Builder module with the following exceptions:
    • DB2 files with direct access residing in a different DB2 subsystem from the one in which the product is installed.
    • DL/I files with direct access
    Note: In these two cases, before creating the extraction method, you must specify a sequential unload of the data stores and insert the Process Identifier for reading the data stores using the jobs Load DB2 Table Information by Unload or Load DL/I Database Information. See the Front End Guide for more information.
Parent topic: Data Subset Extraction Guide


Getting Started

This chapter provides an overview of the Data Subset Extraction module. You should read it before using the module.
  • General characteristics of use
  • Starting Data Subset Extraction
  • Menus
  • Context menus
Parent topic: Data Subset Extraction Guide


General characteristics of use

Data Express for z/OS can be used in either Standard or Client/Server mode, whereas Data Express for Distributed Systems can only be used in Standard mode:

Data Express can be used in either Standard or Client/Server mode:

  • Standard - Supports direct access to the data resident on RDBMS from your PC.
  • Client/Server - Supports access to the data resident on RDBMS through the Data Express kbde-Server: a product based on a three-tier architecture where the second tier handles data transfer optimization. Client/Server mode lets you access data over the Internet with the lowest amount of administration necessary and the best utilization of network capacity.
    Note: Client/Server mode can use the same functions as Standard mode without you having to install additional software on your PC, such as the DB2 client or the Borland Database Engine (BDE).

For Data Subset Extraction configuration information and instructions, see:

  • Appendix A Standard Mode in the Front End Guide.
  • Appendix B. Client/Server Mode in the Front End Guide.

The following guidelines apply to windows and screen items within the Data Subset Extraction module:

  • The main window, Data Subset Extraction - [Master Form], cannot be closed except when exiting the module.
  • Multiple MDI windows may be opened simultaneously.
  • Right-click an item or area to view a list of applicable commands.
Parent topic: Getting Started


Starting Data Subset Extraction

To start Data Subset Extraction:

  1. Click Start > All Programs > Micro Focus Data Express 4.0 > Data Subset Extraction.
    Note: If you are using Data Express on Windows Vista and User Access Control is enabled, you must run Data Subset Extraction as Administrator.
  2. Select the required database with which to launch the connection. By default, the highlighted item in the list of available databases is the last database you connected to using the Data Subset Extraction module.
    Note: To connect to the database for first time, you need your user ID and password.
  3. Select the required schema. The Data Subset Extraction - [Master Form] window appears:
    Figure 1. Figure 4-1: Data Subset Extraction - [Master Form] Window

Tip: Once you are connected to a database, you can open and close a database from the Data Subset Extraction - [Master Form] window by clicking File > Open or File > Close.
Parent topic: Getting Started


Menus

This section describes the menus and common toolbars displayed in the Data Subset Extraction - [Master Form] window.
File menu
The File menu contains the following commands:
  • Open - Opens a new database connection.
  • Close - Closes the database connection.
  • Database - Schema - Indicates the most recently opened database.
  • Exit - Terminates the Data Subset Extraction module.
Tools menu
The Tools menu contains the following commands:
  • Options - Opens the Data Subset Extraction Option window for setting the help language, security options, and a directory for storing the temporary files generated using the New Method wizard.
Environment menu
The Environment menu contains the following commands:
  • Create new Method - Launches the New Method Wizard so that new methods can be created.
  • Work with selected Method - Opens the Work with Method window containing information for the selected method in order to update or view information.
  • Delete selected Method - Deletes the selected method.
  • Work with Groups - Opens the Work with Groups window where you can update existing groups and carry out maintenance operations.
  • Work with selected Group - Opens the Work with Group window containing information for the selected group in order to update or view information.
  • Refresh - Refreshes information.
View menu
The View menu contains the following commands:
  • Toolbar - Displays or hides the toolbar.
  • Status bar - Displays or hides the status bar.
Window menu
The set of open windows can be organized according to the standard Windows arrangements. The Window menu contains the following commands:
  • Minimize All - Minimizes all open windows with the exception of the main window.
  • Arrange - Arranges all minimized windows.
Help menu
The Help menu contains the following commands:
  • Help Topics , Index - Opens online help information for the product in the language selected in the Options window.
  • Micro Focus SupportLine - Contains commands for connecting to Micro Focus Product Support for information or technical assistance.
  • Micro Focus - Data Home Page - Connects to the Micro Focus Data Express website.
  • About Data Subset Extraction - Displays production information for the Data Subset Extraction module.
Parent topic: Getting Started


Context menus

This section describes the commands included in context menus.
  • Column sorting
  • Grid functions
Parent topic: Getting Started


Column sorting

A context menu to display a list of sort options can be displayed by right-clicking a column header.

Note: All changes to the grid layout are saved in the Windows system registry. For example, groups , column width attributes, column order attributes , and column font and color information . However, filtering and sorting preferences are not saved.

The column sorting context menu contains the following commands:

  • Sort Descending - Sorts the list contents in descending order. The same result can be obtained by simply clicking the column heading.
  • Sort Ascending - Sorts the list contents in ascending order. The same result can be obtained by simply clicking the column heading.
  • Group by This Column - Groups the list according to the contents of the selected column if the Group Panel has been enabled.
  • Remove This Column - Deletes the selected column.
  • Column Selector - Displays a Customize window that contains a list of columns that have been removed previously. A column can be restored by performing a drag-and-drop operation to the desired position in the list. A column can be removed from a list by performing a drag-and-drop operation from the list to the Customize window.
  • Alignment - Aligns the content of the columns to the center, right, or left as specified by the user.
  • Best Fit - Resizes the selected column to allow the content of each record to be displayed. The same result can be obtained by double-clicking the column's external margin.
  • Pick Color - Allows the color of the selected column to be changed.
  • Pick Font - Allows the font, font style, size, color, and script to be changed.
  • Best Fit (All Columns) - Resizes all the columns from the list to allow the content of each record to be displayed.
Note: Columns can also be sorted by clicking or performing a drag-and-drop operation on the column heading. Information displayed can be reduced by clicking a column heading, and then typing the first letter of the items you are interested in.
Parent topic: Context menus


Grid functions

A context menu to display a list of functions for a grid can be displayed by right-clicking anywhere on the grid.

Note: All changes to the grid layout are saved in the Windows registry. For example, groups , column width attributes, column order attributes , and column font and color information . However, filtering and sorting preferences are not saved.

The grid functions context menu contains the following commands:

  • Column Selector - Displays a Customize window that contains a list of columns that have been removed previously. A column can be restored by performing a drag-and-drop operation to the desired position in the list. A column can be removed from a list by performing a drag-and-drop operation from the list to the Customize window.
  • Best Fit (All Columns) -Resizes all the columns from the list to allow the content of each record to be displayed.
  • Export to File… - Exports the information to a file using one of the following compatible formats:
    • Comma separated value (.csv)
    • Hyper Text Markup Language (.htm)
    • Text (.txt)
    • Excel (.xls)
    • Extensible Markup Language (.xml)

  • Show Summary Footer - Displays the summary footer beneath the grid. By selecting a row and right-clicking the summary footer, a context menu displaying arithmetic operations you can perform on the selected row appears.

  • Show Group Panel - Displays the column name by which data is grouped above the grid. By dragging the column headers into this window, a group hierarchy can be defined.
    Note: If data is being grouped by column, a footer panel is displayed beneath the grid for each group.
  • Filter Active - If enabled, lets you choose data to be displayed on the grid; a downward arrow icon will appear in the column headers.

    Click an arrow to choose the data to be displayed on the grid. This function lets you generate a personal filter.

Note: The time taken to display the window increases with the number of records to be loaded.
Parent topic: Context menus


Introduction to Data Subset Extraction window

After starting the Data Subset Extraction module, the main window Data Subset Extraction - [Master Form] appears.

From this window, you can view information regarding existing groups and methods.

  • Window components
  • Node pane
  • Column headers by group
  • Column headers by creator
Parent topic: Data Subset Extraction Guide


Window components

This section describes each area of the Data Subset Extraction - [Master Form] window.

The Data Subset Extraction - [Master Form] window comprises two main areas: the node pane and the grid area as shown:

Figure 1. Figure 5-1: Data Subset Extraction - [Master Form] Window

Parent topic: Introduction to Data Subset Extraction window


Node pane

The Node pane contains a hierarchical structure that shows the grouping associated with each root-level node.

The two root-level nodes are Groups and Creators.

  • Icons
  • Context menu
Parent topic: Introduction to Data Subset Extraction window


Icons

Below the hierarchical structure are two icons (By Group and By Creator) that let you sort information by altering the display of the root node and sub nodes. By default, the hierarchy is sorted by groups.

A. By Group

If the By Group icon is specified and the Groups root node is selected, the list of groups included in the environment is displayed in the List of Groups grid, whereas the grid in the lower part of the screen, List of Methods in Group, displays the methods belonging to the group for the selected record. By expanding the Groups root node, the list of the environment groups is visible.

If one of the groups is selected, the List of Methods in Group grid containing environment methods is displayed in the upper part of the screen, whereas the grid in the lower part of the screen, List of Steps in Method, displays the steps making up the method of the selected record.

B. By Creator

If the By Creator icon is specified and the Creators root node is selected, the list of creators included in the environment is displayed in the List of Creators grid, whereas the grid in the lower part of the screen, List of Methods created by, displays the methods defined by the creator belonging to the selected record. By expanding the Creators root node, the list of the environment creators is visible.

If one of the creators is selected, the List of Methods created by grid containing environment methods is displayed in the upper part of the screen, whereas the grid in the lower part of the screen, List of Steps in Method, displays the steps making up the method of the selected record.

Parent topic: Node pane


Context menu

A context menu to display a list of functions can be displayed by right-clicking anywhere in the node pane.

This context menu contains the following commands:

  • Work with selected Method - Displays the Work with Method window including the details for the selected method.
  • Save Method As - Displays the Save Method As window where the method name and description can be changed.
  • Create new Method - Launches the New Method wizard.
  • Delete selected Method - Deletes the selected method.
  • Work with selected Group - Displays the Work with Group window for the selected group.
  • Refresh - Updates information in the node pane corresponding grid areas.
Parent topic: Node pane


Column headers by group

This section describes the column headings in the Grid area for data stores that have been loaded in the Data Express Knowledge Base when items in the Node pane have been sorted by group.
  • List of groups
  • List of methods in group
  • List of Steps in Method Group
Parent topic: Introduction to Data Subset Extraction window


List of groups

The following list describes the column headings in the List of Groups grid area when the Groups root node is selected:

  • Name - Name of the group.
  • Description - Description of the group
  • Machine ID - Identifier of the machine.
  • Company - Identifier of the company.
Parent topic: Column headers by group


List of methods in group

The following list describes the column headings in the List of Methods in Group grid area for the method selected in the List of Methods in Group grid, when the Groups root node is selected.

  • Method - Name of the method.
  • Description - Description of the group.
  • Machine ID - Identifier of the machine.
  • Company - Identifier of the company.
  • Group name - Name of the group.
  • Creator - Name of user creating the method.
  • Elaboration Date - Method elaboration date.
  • Active - Method activated (Yes activated, No deactivated).
  • Completed - Method completed (Yes completed, No not completed).
Parent topic: Column headers by group


List of Steps in Method Group

A. Method Properties

The following list describes the column headings in the List of Steps in Method grid area (from the Method Properties tab) for the method selected in the List of Methods in Group grid, when either a group node or a method node is selected:

  • Method - Name of the method.
  • Step - Number of the method step.
  • Description - Description of the step.
  • Elaboration - Elaboration state of the step (Y executed, N not executed).
  • Elaboration Date - Step elaboration date.
  • Elaboration Time - Step elaboration time.

B. Method Overview

The Method Overview tab is only visible if the batch job has been already executed for this method.

The bar chart shows the number of the data stores in the selected method and their original and reduced sizes as follows:

  • Reduced Data Stores - Data stores containing the filter class.
  • Not Reduced Data Stores - Data stores that do not contain the filter class.
  • All Data Stores - Sum of the Reduced Data Stores and Not Reduced Data Stores.
Parent topic: Column headers by group


Column headers by creator

This section describes the column headings in the Grid area for data stores that have been loaded in the Data Express Knowledge Base when items in the Node pane have been sorted by creator.

  • List of creators
  • List of methods created by
  • List of steps in method creator
Parent topic: Introduction to Data Subset Extraction window


List of creators

The following list describes the column heading in the List of Creators grid area when the Creators root node is selected :

  • Creator - Name of the creator.
Parent topic: Column headers by creator


List of methods created by

The following list describes the column headings in the List of Methods created by grid area when either a creator node or a method node is selected:

  • Method - Name of the method.
  • Description - Description of the group.
  • Machine ID - Identifier of the machine.
  • Company - Identifier of the company.
  • Group Name - Name of the group.
  • Creator - Name of user creating the method.
  • Elaboration Date - Method elaboration date.
  • Active - Method activated (Yes activated, No deactivated).
  • Completed - Method completed (Yes completed, No not completed).
Parent topic: Column headers by creator


List of steps in method creator

The following list describes the column headings in the List of Steps in Method grid area for the method selected in the List of Methods created by grid, when either a creator node or a method node is selected:

  • Method - Name of the method.
  • Step - Number of the method step.
  • Description - Description of the step.
  • Elaboration - Elaboration state of the step (Y executed, N not executed).
  • Elaboration Date - Step elaboration date.
  • Elaboration Time - Step elaboration time.
Parent topic: Column headers by creator


Work with Groups

The Work with Groups window lets you to define, view, modify, and delete groups.

Groups can be used to logically separate the methods created for each Machine ID and Company. Each method must belong to a group.

  • Introduction to Work with Groups
  • Window contents
  • Available operations
Parent topic: Data Subset Extraction Guide


Introduction to Work with Groups

To access the Work with Groups window, click [image] (Work with Groups) from the main window.
Parent topic: Work with Groups


Window contents

This section describes the columns and fields listed in the Work with Groups window as shown below:

Figure 1. Figure 6-1: Work with Groups Window

  • Name - Identifier of the group. The information in this column matches the information in theGroup Name field.
  • Description - Description of the group. The information in this column matches the information in theGroup Description field.
  • · Machine ID - Identifier of the machine.
  • Company - Name of the company.
Parent topic: Work with Groups


Available operations

The buttons at the bottom of the Work with Groups window are:

  • Refresh - Refreshes the information listed in the grid. If another user creates a group from a different executable in the same database, it is possible to see the changes.
  • Apply - Confirms the changes made to the selected group. Only the Group Description field can be modified.
  • New - Initializes fields for insertion when creating a new group.
  • Delete - Deletes the selected group if it is not associated with any method.
    Note: All methods contained within the group must first be deleted in the Work with Methods window.
  • Cancel - Cancels any changes made to the selected group. Any actions confirmed with the Apply button cannot be restored with Cancel but must be entered and confirmed again.
  • OK - Closes the window.
Parent topic: Work with Groups


New Method wizard

The New Method wizard takes user input to generate a new method, which is the extraction proposal containing the set of operations needed for the creation of the test environment. The method is later completed in the Work with Method area.
  • Introduction to New Method wizard
  • Creating a new method
  • Available operations
Parent topic: Data Subset Extraction Guide


Introduction to New Method wizard

The Method Creation phase begins with the New Method wizard. During this phase, all the information concerning the method is saved locally in a directory indicated by the user. In the subsequent phase, method confirmation, the method is stored in the Knowledge Base where it is visible and editable by all users.

Reduction can be obtained for data stores of a certain size and/or a particular type. In this case, you must be knowledgeable about the application to determine the appropriate selections. However, the Data Subset Extraction module can recognize and automatically create a first subset containing a series of data stores (called register table data stores) that are initially brought into the new test environment. Identifying register table data stores has the following advantages:

  • You can create a method to extract only the register table data store subset, which is normally valid for the creation of any test environment independently of the selection criteria applied. Once this method has been saved and confirmed, it can be run whenever necessary.

  • You can create a second data store subset, which filters the register data store subset you created, to produce a data store subset that is reduced even further.

To access the New Method wizard, click [image] (Create New Method) from the main window .

Parent topic: New Method wizard


Creating a new method

This section explains how to create a new method using the New Method wizard.

Before creating a method, use Data Builder to complete the following operations:

  1. Load and analyze the application copybooks.
  2. Load and analyze the data stores.
  3. Associate data stores and copybooks automatically.
  4. Define and associate the data elements with classes needed for the extraction. For example, define the primary selection class and the related class. Other classes (item class) can be defined and assigned to perform further extraction operations.
  5. Make sure that the data stores to be processed are correctly split into the two categories, Register Tables and Others.

For more information about Data Builder, see the Front End Guide .

To define a method with the New Method wizard, you must complete 3 steps:

  1. Definition of general method characteristics: Inclusion/exclusion of Register Tables data stores.
  2. Definition of the primary extraction criteria: Identification of a selection class to divide other data stores into class-sensitive and non-class-sensitive data stores.
  3. Definition of relationship criteria: Identification of a connection (foreign key and/or primary key) between related (class-sensitive and non-class-sensitive) data stores.
Note: The Next buttons are not enabled if an error occurred while entering data.
  • Definition of general method characteristics
  • Definition of primary extraction criteria
  • Definition of relationship criteria
Parent topic: New Method wizard


Definition of general method characteristics

To define general method characteristics:

  1. Click [image] (Create new Method) to display the New Method wizard as shown below:
  2. Specify the Machine ID and Company for the data stores that must be processed.
    Note: Data stores belonging to different machine IDs and companies cannot be included.
  3. Click Next to go to the next screen.
    Figure 1. Figure 7-2: New Method Wizard - Set Method Properties

  4. Specify the Group name, Method, Method description, and Creator.
    Note: If a group has not previously been defined in the Work with Groups area, it can be defined from this screen, by simply specifying its name and description in a specific field.
  5. Click Next to go to the next screen.
  6. Indicate the type of files you want to include in the method by checking one or more of the following:
    • Import all register table Data Stores - Demographic data stores are registered in a dedicated elaboration step.
    • Import all non-register table Data Stores.
  7. If only Import all register table Data Stores is checked, click Finish. Otherwise, continue to the next section.
Parent topic: Creating a new method


Definition of primary extraction criteria

This procedure lets you indicate selection classes, which are classes that you want to apply the filter on in order to obtain a subset of data stores.

To define the primary extraction criteria:

  1. Click Next to go to the next screen.
  2. Select one or more classes that will be used as the primary extraction criteria.

    The Data Subset Extraction module automatically extracts all the environment files containing the selected classes, entering them into a specific step. Once the method has been created, the remaining data stores (that is the data stores that are not sensitive to the selected classes) will initially be excluded from elaboration. During the Method Confirmation phase, they can be included in the method again. These data stores will then be integrally copied into the new environment generated by the method involved.

  3. Continue to the next section.
Parent topic: Creating a new method


Definition of relationship criteria

The Data Subset Extraction module automatically identifies all the data stores associated with a related class and includes them in specific subsets. The procedure lets you indicate related classes, which identify the subsets containing all the files associated with one of the identified classes. To subset the data, a filter is applied based on a list/range of values so that all data stores can be automatically copied according to the specified criteria.

Files correlating to those previously filtered may be included in this last subset. These data stores can contain independent information items and, therefore, be integrally copied to maintain the integrity of data. They can also contain strictly correlated information items, such as additional data or document details. Their integral copy will not ensure that data is integrally maintained; that is, the system will include information whose primary origin has not been maintained.

To define the relationship criteria:

  1. Click Next to go to the next screen.
  2. One or more classes related to each class selected as a primary extraction step can be indicated. The primary extraction criteria is listed in the Selection class field.
  3. Choose the appropriate class in the field Selection class and use Add to relate additional classes in the Other Class list.

    The Data Subset Extraction module automatically extracts all the environment files containing the related classes and inserts them in a further elaboration step.

  4. Click Finish. The Create Method window is displayed.
  5. If required, modify the recommended path where method information will be stored locally, and click Start.
  6. Click OK in the Generating files list window to open the Work with Method area.
  7. Complete the method by confirming it and by cataloging data stores into elaboration steps in the Work with Method area. For more information, see Work with Method.
Parent topic: Creating a new method


Available operations

The buttons in the New Method wizard are:

  • Add - Adds the selected classes in the Other Class list to the Selection Class list.
  • Remove - Removes the selected class from the Selection class list.
  • Refresh - Refreshes the information listed for all created classes.
  • Previous - Goes back to the previous page in the New Method wizard.
  • Next - Advances one page in the New Method wizard.
  • Finish - Updates the various tables with information for the new method and saves the elaborations for the new method in a temporary file.
  • Abort - Exits the New Method wizard without saving changes.
  • Restart - Restarts the New Method wizard without saving changes.
Parent topic: New Method wizard


Work with Method

The Work with Method area lets users confirm newly created methods or modify existing methods at any point after they have been created using the New Method Wizard .

Each newly created method must be completed during the Method Confirmation phase.

  • Introduction to the Work with Method area
  • Fields
  • Steps context menu
  • Tabs
  • Available operations
  • Method confirmation
Parent topic: Data Subset Extraction Guide


Introduction to the Work with Method area

If you have just created a method using the New Method Wizard, the Work with Method area is automatically displayed for the method being defined. Otherwise, to access the Work with Method area, perform the following steps:
  1. Launch Data Subset Extraction and access the main window.
  2. Locate the method by searching by group or creator in the node pane of the main window. (See the Node pane section.)
  3. Select the method.
  4. Right-click Work with selected Method.

    The Work with Method window comprises a general fields area, the Steps pane, and the following tabular views:

  • Method Confirmation phase
  • Method elaboration
  • Method Activation phase
  • Extraction Batch Job Launch phase
Parent topic: Work with Method


Method Confirmation phase

During the Method Confirmation phase, specified filters are applied to the primary classes needed to perform the extraction, and the method information is imported from local databases. Each data store subset is cataloged in a specific elaboration step that can occur during any phase.

Before you can confirm the method, you must specify all required connections needed to obtain a further division of data stores into subsets, and you must resolve the problems highlighted in the Warnings tab.

Parent topic: Introduction to the Work with Method area


Method elaboration

Method elaboration occurs after the Method Confirmation phase, but can also occur during or after the Method Creation, Method Confirmation, or Method Activation phase. To complete the method, you must decide whether the files that were initially excluded from elaboration during the method creation phase should continue to be ignored by the method, or if the data stores should be included.

After the method has been saved and confirmed, the extraction batch process elaborates steps sequentially. The elaboration of each step can exploit the information obtained during the elaboration of the preceding step. During method elaboration, the following events occur sequentially:

  1. Elaboration and integral copy of the data stores recognized as Register Tables.
  2. Elaboration of the data stores recognized as sensitive to a selection class, applying the filter specified by the user and copying only the information complying with the required selections. If a related class is included in these data store, all the values of the field associated with the related class are stored in specific lists for all the copied records (in this phase, the related class is also called out class or output class).
  3. Elaboration of the data stores recognized as sensitive to a related class, applying the lists of values generated in the preceding step as filter and copying only the information that comply with the data contained in the lists. (In this phase, the related class is also a selection class.)
  4. Elaboration and integral copy of the remaining data stores.
Note: The flow described above is confined to a single level of relationships. Normally, there can be connections and interdependence between data stores at several levels.
Parent topic: Introduction to the Work with Method area


Method Activation phase

Method activation occurs after the method has been created and confirmed, and most step elaboration has been completed. For instructions, see Method activation in the Work with Method method properties section.
Parent topic: Introduction to the Work with Method area


Extraction Batch Job Launch phase

Methods are created entirely by the client side of Data Subset Extraction. For the Data Express for z/OS solution, the extraction batch job using the created method is performed by the z/OS platform, though the job is launched from the client side. For more information, see chapter Work with Jobs in your Front End Guide.

Before launching the extraction job, you must have allocated the output sequential files with names corresponding to those of the output files indicated during the method confirmation phase on the client side.

Parent topic: Introduction to the Work with Method area


Fields

The upper part of the window displays fields that contain information for selected method.

The fields listed in the upper part of the Work with Method area are:

Machine ID
Identifier of the machine on which the method is operating.
Group
Name and description of the group to which the method belongs.
Company
Identifier of the company on which the method is operating.
Method
Name and description of the method.
Creator
Name of the user creating the method.
Date
Execution date of the method.
Active
Indicator of whether the method has been enabled.
Completed
Indicator of whether the method has been completed during the confirmation phase.
Records Limit
The maximum number of records that the Data Subset Extraction process can generate.
Parent topic: Work with Method


Steps context menu

This section describes the commands listed by right-clicking in the Steps pane of the Work with Method area.

The steps context menu contains the following commands:

  • Create new Step - Creates a new step with a proposed step number that is 10 digits greater than the previous step. A number between two existing steps can be used as the step number.
  • Work with Step - Opens the Work with Step window to activate the modification phase for the selected step. Both the step description and the linked documentation file can be modified.
  • Delete Step - Deletes the selected step. This function is activated only for the steps that do not have assigned elaborations.
Parent topic: Work with Method


Tabs

This section describes each tab of the Work with Methods area. The tabs are:
  • Elaborations
  • Excluded files
  • Method overview
  • Warnings
Parent topic: Work with Method


Elaborations

The Elaborations tab includes the list of elaborations (without incongruence) that process register table files, files containing the primary class, and files containing related classes.

The section describes the column headings and context menu commands accessible from the Elaborations tab.

  • Column headings tabs
  • Context menu tabs
Parent topic: Tabs


Column headings tabs

The columns listed on the Elaborations tab of the Work with Method area are:

  • Step - Number of the step.
  • Data Store Name - Name of the data store.
  • Record format - Format of the record.
  • Selected Class Description - Description of the selected class.
  • Selected Field Name - Name of the field associated with the input class.
  • In Filter - Type of filter applied to the selected class (input class).
  • Out Class Description - Description of the output class.
  • Out Field Name - Name of the field associated with the output class.
  • Output Filter - Generated filter code (of the List type).
  • Write File - Indicator of whether an output file must be created (Y create output file, N do not create output file) for the current elaboration.
  • Elaborated - Indicator of whether the file has been elaborated.
  • Number of Written Records - Number of records written on the output file.
  • Number of Read Records - Number of records read by the extraction process.
  • Reduction in Bytes - Number of bytes of the last reduction process (in simulation or in real).
  • Reduction in Percentage - Percentage of the last reduction process (in simulation or in real).
  • CPU Time (hh:mm:ss) -CPU time of the last reduction process (in simulation or in real).
  • Elapsed Time (hh:mm:ss) - Elapsed time of the last reduction process (in simulation or in real).
  • Simulation Date - Date when the last simulation extraction ran.

  • Real Extraction Date - Date when the last real extraction ran.

Parent topic: Elaborations


Context menu tabs

This section describes the commands listed by right-clicking a selected file in the Work with Method area from the Elaborations tab. The context menu contains the following commands:

  • Move Selected to - Moves the selected elaboration to the indicated step. A list of the existing steps is displayed.

  • Add from Excluded Data Stores - Displays a list of data stores not included in the method that contain the same class specified as the output class of the selected data store. The output filter generated by the selected data store's output class becomes the input filter for those data stores. You can select one or more data stores that will be included in the step following the one of the selected elaboration. If there is no subsequent step, its creation will be automatically requested.

  • Elaboration
    • Clear Selection Class - Deletes the elaboration input class.
    • Clear Output Class - Deletes the elaboration output class.
    • Clear Output and Selection Class - Modifies the input and output class.
    • Selection class / Filter properties - Displays the List of Class Elaboration window in order to choose other filter settings for the selected class.
    • Filter Settings -Sets, modifies, or displays selection filters on classes.
    • Copy Filter from - Applies the filters already defined for the same class in the method to a class included in the file. Unlike the Connect Existing Filter command, this associates the class with a copy of the filter associated with a class. Therefore, any modification to this filter will have no impact on the other classes.
    • Set/Reset Elaborated - Allows you to change the elaboration status, so you can reset the elaboration flag in order to allow a new elaboration on the same file.
      Note: This is related to the Run deletion parameter. If you run an extraction with Run deletion active, all files included in the parameter range are elaborated. If Run deletion is not active, only files having elaboration flag not set are elaborated.

    • Delete Elaborated - Deletes the elaboration of the file.

  • Elaboration Properties - Displays the Elaborations Properties window containing general information on the elaboration of the file and on the method. From this window, you can modify the name of the writing program, the name of the selection program of the records for the file, and the name of the output file.
  • Create New List - Verifies the existence of one or more classes that can be used as output class in the elaborated file. If the search is successful, it proposes the list of classes.
  • Create New List Using Existing Filter - Displays the List of Classes and Filters assigned to the Current File window. Choose class and filter to use for this new list.
  • Create Output Data Store - Writes the output data store for the selected elaboration. All elaborations have this setting by default.
  • Don't Create Output Data Store - Indicator of whether to write the output data store for the selected elaboration.
  • Utility - Deletes unused filters.
Parent topic: Elaborations


Excluded files

The Excluded Files tab contains the list of files that have been temporarily excluded from expansions, Excluded files do not contain primary or related classes. Files to be involved in new expansions can be taken from this list.

The section describes the column headings and context menu commands accessible from the Excluded Files tab.

  • Column headings
  • Context menu
Parent topic: Tabs


Column headings

The columns listed on the Excluded Files tab of the Work with Method area are:

  • Step - Number of the step.
  • Data Store Name - Name of the data store.
  • Data Store type – Type of data store. For example, DB2, VSAM, DL/I, GDG, SEQ, ODBC, or ORA.
  • Record format - Format of the record.
Parent topic: Excluded files


Context menu

This section describes the command listed by right-clicking a selected data store in the Work with Method area from the Excluded Files tab. The context menu contains the following command:

  • Move Selected To - Moves the selected expansion to the indicated step. A list of the existing steps is displayed.
Parent topic: Excluded files


Method overview

The Method Overview tab displays general method information and graphics presenting the number of files included in the selected method. This tab is visible only if the extraction batch job has been already completed for this method.

The pie chart shows the distribution of the classes contained in the files of the selected method. By default, it shows the number of files containing classes and those that do not contain any classes.

The bar chart shows the number of the files in the selected method and their original and reduced sizes for the reduced and not reduced files. The Reduced Files are files containing the filter class, Not Reduced Files are files that do not contain the filter class, and All Files is the sum of Reduced Files and Not Reduced Files.

The columns listed on the Method Overview tab of the Work with Method area are:

  • Method Name - Name and description of the method.
  • Group - Name and description of the group to which the method belongs.
  • Machine ID - Identifier of the machine on which the method is operating.
  • Company - Identifier of the company on which the method is operating.
  • Number of Data stores/tables - Number of the files or tables included in the method.
  • Size (in GBs) - Reduced size in gigabytes.
Parent topic: Tabs


Warnings

The Warnings tab displays the list of elaborations (with incongruence) or ambiguities that must be resolved by the user. For example, an elaboration for a file containing more than one selection class will generate a warning as there can be only one input class for the same file in the same step.
Parent topic: Tabs


Available operations

The buttons at the bottom of the Work with Method window are:

  • Properties - Displays the Method Properties window containing general information about the method. From this window, you can edit the method description, the Method Active state and the path of the Documentation File.

  • Refresh - Refreshes the window.
  • Copy to Next Step - Carries the selected expansion forward to the next step.
  • Exclude Elaboration - Deletes the selected expansion. If the file to be deleted is not included in other steps, it is copied in Excluded Files (if the list is active).
  • Confirm - Confirms the method. The expansions stored on the local file are transferred to an effective database.
  • OK - Saves any changes that have not been applied and closes the window.
Parent topic: Work with Method


Method confirmation

This section describes the procedure for confirming a new method. Method confirmation lets you:

  • Provide information concerning the names that will be used for the output files of the new environment. During the method confirmation phase, the tool allows you to adopt a single naming mode, which will be set for all the environment files. Once a method has been confirmed, the name of the output file of an expansion can be modified from the Elaboration Properties window.
    Note: For MVS databases, the output file name corresponds to the specified file, not to the step. If you specify the same file in multiple steps, only one output file is generated for the specified file regardless of how many steps specify it.
  • Define filters on the selection classes. Be aware that, when defining filters, you cannot define more than one input or one output filter with the same class in a single step. You must use a different class for each filter.
To confirm the method:
  1. In the Work with Method area, click Confirm. The Confirm Method window is displayed.
  2. Assign a naming convention to output files by choosing one of the following options:
    • Add Prefix - Adds the prefix specified in the Prefix box to the name of the data set for the output file and to the name of the data set for the input file.
    • Add Suffix - Adds the suffix specified in the Suffix box to the name of the data set for the output file and to the name of the data set for the input file.
    • String Replacement - Generates a new data set name for the output file by replacing a part of the data set name the input file with the text specified in the String to search box with text in the Replace with box.
  3. If required, check Move excluded files into new step to terminate the creation of the method inserting all the files excluded from the elaborations in the final step of the method.
  4. Click OK to save your changes or click Cancel to discard your changes.
  5. If no errors occurred, apply filters to the primary selected class in the Selection class / Filter properties window as required (see the chapter Work with Method - Selection class / Filter properties). Otherwise, errors have occurred while output file names were automatically assigned, and the names of the files must be manually assigned. The following conditions will generate errors:
    • The file name exceeds 44 characters.
    • Characters to be replaced have not been found in the input name.
    • One or more qualifiers exceed 8 characters.

    When errors occur, the Reconcile window is displayed.

  6. Edit the file name for each output file listed in the Reconcile window. To edit the file name for a single file, select the file, type the new name in the Unload Output File Name window, and click OK. To edit all file names, click Modify to access the file naming definition window and edit the criteria as required.
Parent topic: Work with Method


Work with Method selection class/filter properties

During the Method Confirmation phase, you can apply filters to the primary selected class in the Selection class / Filter properties window.

However, the Selection class / Filter properties window lets you modify the filters applied to a class even after the method has been confirmed or insert new filters if selection classes are modified or added.

  • Introduction to selection class/filter properties
  • Window contents
Parent topic: Data Subset Extraction Guide


Introduction to selection class/filter properties

To display the Selection class / Filter properties window, select the appropriate step in the Work with Method area. Right-click to display the context menu, point to Elaboration, and then click Selection class / Filter properties.
Parent topic: Work with Method selection class/filter properties


Window contents

This section describes the columns, fields, and filter types listed in the Selection class/ Filter properties window.

  • Column headings in top grid
  • Global filter fields
  • Filter types
Parent topic: Work with Method selection class/filter properties


Column headings in top grid

The List of class in elaboration grid displays the list of primary selection classes needed for method confirmation.

The following list describes the columns listed in the List of Class in Elaboration grid of the Selection class / Filter properties window:

  • Class Code - Class identifier code.
  • Class Name - Name of the class.
  • Data Element name - Name of the data element.

  • Data Element Type - Types of data allowed for the data element.

Parent topic: Window contents


Global filter fields

The following list describes the fields listed for each filter type in the Selection class / Filter properties window:

  • Filter description - Description of the filter.

  • Documentation path - File annexed to the filter for documentation purposes.

Parent topic: Window contents


Filter types

This section describes the filter types that can be applied to the primary classes listed in the List of Class in Elaboration grid in the Select class/ Filter properties window.

If you want to apply a filter to a class and there are several primary selection classes, you must select each class from the list and later select the type of filter you want to specify. This operation must be repeated for each class in the list for which you want to apply a filter.

Note: The filter values for date fields must be in the format YYYYMMDD or YYYYMMDD HHHH.MM.SS where HHHH is based on a 24-hour clock.

After associating the correct filter with each class, click OK.

  • Filter type properties
Parent topic: Window contents


Filter type properties

NO FILTER
Selecting the filter type NO FILTER does not display additional fields.
FILTER BY RANGE

Displays the following:

From Value
Start value to be searched for.
To Value
End value to be searched for.
Range Mode
Values in range are included
Considers values within the specified range.
Values in range are excluded
Considers values outside the specified range.
FILTER BY USER EXIT

Displays the following:

User Exit name
Name of the user program performing the selection.
FILTER BY LIST VALUE

Displays the following:

Values List
One or more values to be selected.
List Mode
Values in list are included
Considers values in the list.
Values in list are excluded
Considers values outside the list.
FILTER BY FILTERED LIST

Selecting the filter type FILTER BY FILTERED LIST does not display additional fields.

FILTER BY EXISTING FILTER
Selecting the FILTER BY EXISTING FILTER filter displays a grid where you can perform drag-and-drop operations to group filters by columns.
FILTER BY NULL

Enables the extraction to select all records having null values for the selected columns. Valid for distributed data stores only.

Parent topic: Filter types


Work with Method work with steps

From the Work with Steps window, you can create new steps for a method and display the characteristics of the existing steps.

  • Introduction Work with Method work with steps
  • Window contents
  • Available operations
Parent topic: Data Subset Extraction Guide


Introduction Work with Method work with steps

The Work with Steps window can be accessed from different areas of the Data Subset Extraction module, in that it may appear while performing standard operations during method modification.

To display the Work with Steps window in display mode, select the appropriate step in the Steps pane of the Work with Method area. See the Introduction to the Work with Method area section. Right-click Work with Step.

To display the Work with Steps window in insert mode, select the appropriate step in the Steps pane of the Work with Method area. See the Introduction to the Work with Method area section. Right-click Create new Step.

Parent topic: Work with Method work with steps


Window contents

This section describes the display and insert modes of the Work with Steps window.

  • Display mode
  • Insert mode
Parent topic: Work with Method work with steps


Display mode

The Work with Steps window in display mode displays information for the selected step. In display mode, only the fields related to the description of the step and to any documentation file associated with it can be modified.

This section describes the Work with Steps window in display mode.

  • Step Number - Number of the step in the method.
  • Step Description - Description of the step.
  • Link to Documentation File - File annexed to the step for documentation purposes.
  • Elaboration Date - Elaboration date of the step.
  • Elaboration Time - Elaboration time of the step.

  • Elaborated - Elaboration state of the step (Y elaborated, N not elaborated).

Parent topic: Window contents


Insert mode

The Work with Steps window in insert mode displays fields that can be defined for the new step you are creating.

This section describes the Work with Steps window in insert mode.

  • Step Number - Number of the step in the method. By default, the tool indicates the progressive number to be associated with the created step. The indicated number is obtained by increasing the number of the last step found in the method by 10.
  • Step Description - Description of the step.

  • Link to Documentation File - File annexed to the step for documentation purposes.

Parent topic: Window contents


Available operations

The buttons at the bottom of the Work with Steps window are:

  • OK - Saves any changes that have not been applied and closes the window.

  • Cancel - Cancels and changes and closes the window.

Parent topic: Work with Method work with steps


Work with Method method properties

The Method Properties window displays all the information concerning the selected method. The method can be activated from this window.

  • Introduction to Work with Method method properties
  • Window contents
  • Available operations
  • Method activation
Parent topic: Data Subset Extraction Guide


Introduction to Work with Method method properties

To display the Method Properties window, click Properties from the Work with Method area. For more information, see Introduction to the Work with Method area.

Parent topic: Work with Method method properties


Window contents

This section describes the fields listed in the Method Properties window.

  • Machine ID - Identifier of the Machine on which the method operates.
  • Company - Identifier of the company on which the method operates.
  • Group - Name and description of the group to which the method belongs.
  • Method - Name and description of the method.
  • Creator - Name of user creating the method.
  • Date - Execution date of the method.
  • Active - Indicator of whether the method has been activated. Selecting the Active checkbox is a prerequisite for the launch of the extraction batch job performed from the z/OS platform. If the check box is dimmed, the method has not been confirmed.
  • Completed - Indicates whether the method has been completed through the confirmation phase.

  • Documentation File - File annexed to the method for documentation purposes.

Parent topic: Work with Method method properties


Available operations

The buttons at the bottom of the Method Properties window are:

  • OK - Saves any changes that have not been applied and closes the window.
  • Cancel - Cancels any changes made to the options. Any actions confirmed with the Apply button cannot be restored with Cancel but must be entered and confirmed again.
Parent topic: Work with Method method properties


Method activation

To activate the method:

  1. From the Method Properties window, check Active. A Warning window is displayed.
  2. Click Yes and then OK to confirm the activation.
Parent topic: Work with Method method properties


Work with Method elaboration properties

The Elaboration Properties window displays all the information concerning the selected elaboration and the file involved in the elaboration.

  • Introduction to Work with Method elaboration properties
  • Window contents
  • Available operations
Parent topic: Data Subset Extraction Guide


Introduction to Work with Method elaboration properties

To display the Elaboration Properties window, select a step in the Files Elaboration list from the Work with Method area. For more information, see Introduction to the Work with Method area. Right-click Elaboration Properties.

Parent topic: Work with Method elaboration properties


Window contents

This section describes the fields listed in the Elaboration Properties window as shown below:

Figure 1. Figure 12-1: Elaboration properties Window

  • Method’s description
  • Elaboration’s description
Parent topic: Work with Method elaboration properties


Method’s description

The following list describes the fields listed in the section Method's Description of the Elaboration Properties window:

  • Group - Name and description of the group to which the method belongs.
  • Method - Name and description of the method.
  • Step - Number and description of the step of the selected elaboration.
Parent topic: Window contents


Elaboration’s description

The following list describes the fields listed in the section Elaboration's Description of the Elaboration Properties window:

  • General Properties
    • Process ID - Identifier of the process associated with the elaborated file.
    • Format Selector - Exit-routine for the recognition of formats associated with the elaborated file.
    • Write Program - Write program of the Output file. This information can be modified.
    • Select Program - Select program of the individual record for the input file. This information can be modified.
  • Unload Input File
    • Data Store name - Name of the Unload Input data store. This is for the z/OS environment only. It can be different from the original name loaded into the Knowledge Base. In order to do this, the process ID associated with the elaboration should be created with characteristics for an Unload File Access (For example: GENUN).
    • Data Store type - Type of the Unload Input data store.
    • Data Store Version - Version of the Unload Input data store.
  • Unload Output File
    • Data Store name - Name of the Unload Output data store.
    • Data Store type - Type of the Unload Output data store. This information can be modified.
    • Data Store Version - Version of the Unload Output data store.
Important: Only the last elaboration (highest step) generates real data.
Parent topic: Window contents


Available operations

The buttons at the bottom of the Elaboration Properties window are:

  • OK - Saves any changes that have not been applied and closes the window.
  • Cancel - Cancels any changes made to the options. Any actions confirmed with the Apply button cannot be restored with Cancel but must be entered and confirmed again.
Parent topic: Work with Method elaboration properties


Work with Method define reload rules

The Define Reload Rules feature is designed to manage the generation of reload JCLs for z/OS DB2 tables that are managed with direct access. Once defined, you can use a reload rule to create reload jobs using Work with Jobs.

  • Introduction to Work with Method define reload rules
  • Window contents
  • Available operations
  • Create reload jobs from reload rules
Parent topic: Data Subset Extraction Guide


Introduction to Work with Method define reload rules

To display the Work with Reload Rules window, click the Reload Rules button at the bottom of the Work with Method window.

Work with Method - Reload Rules
Parent topic: Work with Method define reload rules


Window contents

This section describes the fields listed in the Work with Reload Rules window shown below:

Work with Reload Rules
  • Max no. of Data Stores in JCL - Maximum number of files you want process in a single JCL. If your method exceeds this limit, the function creates a new JCL for the other files. The default value is 2000.
  • Create Syspunch - Indicator of whether or not to include SYSPUNCH in the JCL.
  • Create Reload - Indicator of whether or not to create reload code in the JCL.
  • Target DB2 Owner - Name of the target DB2 owner. If this parameter is not specified, the value is the same as the source DB.
  • Target DSN - Name of the target database subsystem. If this parameter is not specified, the value is the same as the source DB.
  • JCL output data set - Library in which to store generated JCLs.
  • Skeleton for LOAD - Name of the skeleton used to generate reload JCL.
  • Skeleton for REPAIR - Name of the skeleton used to generate repair JCL.
  • Job output member for reload - Name of the generated JCLs used for reload.
  • Job output member for repair - Name of the generated JCLs used for repair.

The following property is specified at the elaboration level:

  • Replace Data - Indicator of whether or not data replacement in the target database is enabled. If not selected, data items are added without deletion.
Parent topic: Work with Method define reload rules


Available operations

The buttons at the bottom of the Work with Reload Rules window are:

  • New - Creates a new rule
  • Delete - Deletes the selected rule
  • Change - Allows the editing of the selected rule parameter
  • OK - Saves any changes that have not been applied and closes the window
  • Cancel - Cancels any changes made and closes the window
  • Apply - Saves any changes

The following operation is available by right-clicking on a selected rule:

  • Assign Elaboration - Include one or more elaboration in the rule by selecting from the list of elaborations that belong to the method

The following operations are available by right-clicking on a selected elaboration:

  • Remove Elaboration from rule - Deletes the elaboration from the rule
  • Replace data - Sets and resets the replace data flag
Parent topic: Work with Method define reload rules


Create reload jobs from reload rules

This function allows you to create JCL to reload all DB2 tables with direct access included in a reload rule belonging to a method.

  • Job creation
  • Available operations
  • JCL information
Parent topic: Work with Method define reload rules


Job creation

To create a Create Reload job:

  1. From the Work with Jobs window, click New.
  2. In the List of Jobs section, select Create Reload Jobs from reload rules.
  3. In the General section, select the appropriate Machine Id and Company name.
  4. In the File Qualification section, specify values for the following fields:
    Application ID
    The application ID, if applicable.
    DB2 Owner, DL/I DBD
    The owner of the DB2 or DL/I DBD file where data is to be masked.
    File type
    The type of file (ADA, VSAM, DB2, DL/I, GDG, SEQ, or ORA).
    File name
    The name of the file containing data to be masked.
    File version
    If the file is GDG, the version number.
    Record format
    Name of the record format.
  5. Click Apply to submit the job. The Secondary Options window appears.
  6. In the Secondary Options window, select the required Group and Method values, and specify values for the following fields:
    From Step / To Step
    Range of values for a subset of steps for the selected method.
    Reload rule
    Name of the reload rule describing how to generate JCL.
    Submit Job
    Indicator of whether to submit the newly created JCLs.
Parent topic: Create reload jobs from reload rules


Available operations

The buttons at the bottom of the Secondary Options window are:

  • Reset All - Resets all parameters to their default values
  • OK - Saves any changes that have not been applied and closes the window
  • Cancel - Cancels any changes made and closes the window
Parent topic: Create reload jobs from reload rules


JCL information

The JCLs are built starting skeletons for which names are defined in reload rules and that can be customized. Working samples are CBTELOD1 for reload and CBTELOD2 for repair; these are the same skeletons that are used by Create reload jobs.

The structure of the skeletons comprises a starting part and an ending part that are repeated once in the created JCLs, and a central part that is delimited by the following keywords:

//* SINGLE FILE ELABORATION and

//* SINGLE FILE END ELABORATION, that is repeated for each file.

In the skeleton, some variables are used and are then changed into the JCL values using the values stored in the Knowledge Base or included in the launch parameters.

The following table describes the used variables:

Variable | Description
&NUM | Numeric progressive
&UNLOADNAM | Name of the sequential data set (output of extraction process)
&OWNAME | Owner of the original table
&FILNAME | Name of the original table
&DBNAME | Database name of the original table
&TSNAME | Tablespace name of the original table
&UNLINPNAM | Name of the unload as specified in the Work with files area
&STEP | Step name for the SYSPUNCH generation (automatically generated)
&STP1 | Step name for the MISYSPUN program call (automatically generated)
&STP | Name of the reload step (automatically generated)
&SYSPARM | It can be ADD,YES, number of sortkeys if no replace parameter, REP,YES, number of sortkeys if replace parameter
&DSNNAME | Name of the input DSN
&METHOD | Name of the method
&JOBN | Name of the job actualized with the JCL name
Parent topic: Create reload jobs from reload rules


Options

Data Subset Extraction allows for product customization in order to meet the requirements for different users.
  • Introduction to Options
  • Window contents
  • Available operations
Parent topic: Data Subset Extraction Guide


Introduction to Options

Options can be edited in the Data Subset Extraction window. To display the Data Subset Extraction window, click Tools > Options. The Options window is shown in the section Window contents.
Parent topic: Options


Window contents

The Options window is divided into pages to help understand the settings.

  • General
  • Server information
Parent topic: Options


General

This section describes the fields listed on the General tab in the Data Subset Extraction window:

  • Help language - The language used for online help. The Help language list only contains the languages available on your PC.
  • Save user and password - An indicator of whether or not you want to save your user ID and password information for the current database connection.
  • Select directory - The directory where unconfirmed methods created with the New Method Wizard are saved.
Parent topic: Window contents


Server information

Connection options to the kbde-Server for client/server mode can be edited in the Data Subset Extraction window. Before opening the project with the Data Subset Extract module, you must define the connection information.

Note: The Server Information tab is displayed only in client/server mode.

The following list describes the fields listed on the Server Information tab of the Data Subset Extraction window:

Figure 1. Figure 13-2: Data Subset Extraction - Server Information Tab

  • IP Address - IP address of the NT server (for DCom) or the Borland socket server (for Socket).
  • URL - URL (for Http) to identify the httpsrvr.DLL file installed on the server machine.
  • Port number - Port number assigned to Borland socket server (for Socket).
  • Connection Options - the type of connection desired (Socket, DCom, Http).
Parent topic: Window contents


Available operations

The buttons at the bottom of the Data Subset Extraction window are:

  • OK - Saves any changes that have not been applied and closes the window.
  • Cancel - Cancels any changes made to the options. Any actions confirmed with the Apply button cannot be restored with Cancel but must be entered and confirmed again.
  • Apply - Confirms the change without closing the window.
Parent topic: Options


Data subset extraction jobs

The Work with Jobs area contains information about the function related to the execution of jobs for test environment generation and is part of the Data Builder module. Jobs are submitted using the Work with Jobs area of Data Builder.

Note: Only the Data Express for z/OS solution makes use of Data Subset Extraction jobs.

For more information about the Work with Jobs area, see Work with Jobs in the Front End Guide.

  • Introduction to data subset extraction jobs
  • Job creation
  • Test environment creation
  • Create allocation jobs
  • JCL information
  • Create unload jobs
    The Create Unload job executes a process to create a JCL that will unload all DB2 tables (included in the method) with direct access that are not involved in masking and reduction routines.
  • Create extraction jobs
    The Create Extraction job executes a process to create a JCL that will execute an extraction method.
  • Create reload jobs
  • Available operations
Parent topic: Data Subset Extraction Guide


Introduction to data subset extraction jobs

To access the Work with Jobs area:

  1. Start Data Builder by clicking Start > All Programs > Micro Focus Data Express 4.0 > Data Builder.
  2. Click [image] (Work with Jobs) in the toolbar of the Data Builder main window.
Parent topic: Data subset extraction jobs


Job creation

This section describes the procedures for creating jobs relevant to Data Subset Extraction.

Parent topic: Data subset extraction jobs


Test environment creation

After all files have been allocated for extraction, launching the Test Environment Creation job is the final step in creating the test environment.

To create a Test Environment Creation job:

  1. Create a new Test Environment Creation job from the Work with Jobs area in the Data Builder module. See Job Creation in the Front End Guide for basic instructions. The Secondary Options window for the Test Environment Creation job is displayed.
  2. Specify the names of the Group and Method from the corresponding drop-down lists. If desired, specify the range of steps for which you want to launch the elaboration.
  3. Specify the remaining information as appropriate:
    • Extraction Type
      • Access Path Simulation - Simulates the path to the output files.
      • Simulation - If this parameter is set, the generated JCL will run the method in simulation, without populating the output files (doing a first run in simulation is always recommended in order to calculate spaces).
      • Real Extraction - If this parameter is set real extraction is performed.
    • Discard Unchanged Files - For DB2 tables only, choose this function to discard from the allocation JCL, all DB2 tables belonging to the current method that are neither reduced nor masked.
    • Run Deletion - If this parameter is set, a deletion of all results of previous extractions for the same step range is performed.
    • Run Extraction - If this parameter is set, the execution of the extractions for the selected step range is performed
  4. Click OK to submit the job.
  5. Wait for the test environment creation phase to be completed.
Parent topic: Data subset extraction jobs


Create allocation jobs

The Create Allocation job executes a process to create a JCL that will delete and allocate all the output for the data set associated with a method or associated with a business protection (masking) procedure.

  • Job creation
Parent topic: Data subset extraction jobs


Job creation

To create a Create Allocation job:

  1. From the Work with Jobs window, click New.
  2. In the List of Jobs section, select Create Allocation Jobs.
  3. In the General section, select the appropriate Machine Id and Company name.
  4. In the File Qualification section, specify values for the following fields:
    • Application ID - The application ID, if applicable.
    • DB2 Owner, DL/I DBD - The owner of the DB2 or DL/I DBD file where data will be masked.
    • File type - The type of file (ADA, VSAM, DB2, DL/I, GDG, SEQ, or ORA).
    • File name - The name of the file where data will be masked.
    • File version - The version number if the file is GDG.
    • Record format - Name of the record format.
  5. Click Apply to submit the job. The Secondary Options window appears:
    Figure 1. Figure 14-2: BTEALSR - Create Allocation Jobs
  6. In the Secondary Options window, select the required Group and Method values, and specify values for the following fields:
    • From Step / To Step - Range of values for a subset of steps for the selected method.
    • Max no. of files in JCL - Maximum number of files you want process in a single JCL. If your method exceeds this limit, a new JCL is created for the other files. The default value of the parameter is 2000.
    • Discard Unchanged Files - Discard all DB2 tables from the allocation JCL that belong to the current method and are neither reduced nor masked.
    • Max cylinder no. for primary quantity - Maximum number of cylinders that can be assigned to a primary quantity. The value of 0 indicates to use the default value of 4300 cylinder.
    • Maximum cylinder No. for secondary quantity - Maximum number of cylinders that can be assigned to a secondary quantity. The value of 0 indicates to use the default value of 4300 cylinder.
    • Maximum size for disk data sets - Maximum number of cylinders (not for the data set) to be allocated on an alternative unit. If the primary quantity exceeds this dimension, the data set will be allocated on an alternative unit.
    • Alternative name for unit - Name of the alternative unit where you want to allocate the data set that exceeded the Maximum size for disk data sets value.
    • Primary quantity percentage - Percentage of cylinder you want to assign at the primary quantity in regard to the total dimension of output data set. The default value is 10%.
    • Secondary quantity percentage - Percentage of cylinder you want to assign at the secondary quantity in regard to the total dimension of primary quantity. The default value is 15%.
    • JCL output data set - Library where the generated JCLs will be stored.
    • JCL output member for deletion - F irst part of the name of the generated JCLs (the second part will be a 3-digit progressive number starting from 001).
    • JCL output member for allocation - First part of the name of the generated JCLs (the second part will be a 3-digit progressive number starting from 001).
    • Submit Job - Indicator of whether to submit the newly created JCLs.
Parent topic: Create allocation jobs


JCL information

The JCLs are built starting from the CBTEALS1 and CBTEALS2 skeletons, which can be customized.

The structure of the skeletons comprises a starting part and an ending part that are repeated once in the created JCLs, and a central part that is delimited by the following keywords:

//* SINGLE FILE ELABORATION and

//* SINGLE FILE END ELABORATION, that is repeated for each file.

In the skeleton, some variables are used and are then changed into the JCL values using the values stored in the Knowledge Base or included in the launch parameters.

The following table describes the used variables:

Variable | Description
&TYPE | Name of the sequential data set (output of extraction process)
&PCYL | Primary quantity
&LRECL | Record length
&UNIT | SYSDA or the value put in the launch screen, depending on the above described rules
&CYL | TRK (if the space is less than a cylinder) or CYL
&SCYL | Secondary quantity
&JOBN | It is actualized with the JCL name
Parent topic: Data subset extraction jobs


Create unload jobs

The Create Unload job executes a process to create a JCL that will unload all DB2 tables (included in the method) with direct access that are not involved in masking and reduction routines.
  • Job creation
  • JCL information
Parent topic: Data subset extraction jobs


Job creation

To create a Create Unload job:

  1. From the Work with Jobs window, click New.
  2. In the List of Jobs section, select Create Unload Jobs.
  3. In the General section, select the appropriate Machine ID and Company name.
  4. In the File Qualification section, specify values for the following fields:
    • Application ID - The application ID, if applicable.
    • DB2 Owner, DL/I DBD - The owner of the DB2 or DL/I DBD file where data will be masked.
    • File type - The type of file (ADA, VSAM, DB2, DL/I, GDG, SEQ, or ORA).
    • File name - The name of the file where data will be masked.
    • File version - The version number if the file is GDG.
    • Record format - Name of the record format.
  5. Click Apply to submit the job. The Secondary Options window appears.
  6. In the Secondary Options window, select the required Group and Method values, and specify values for the following fields:
    • From Step / To Step - Range of values for a subset of steps for the selected method.
    • Max no. filesin JCL - Maximum number of files you want process in a single JCL. If your method exceeds this limit, a new JCL is created for the other files. The default value of the parameter is 2000.
    • Allocate Files - Indicator of whether to place the generated JCL in the allocation of the sequential output data sets.
    • Max Cylinder no. for primary quantity -Maximum number of cylinders that can be assigned to a primary quantity. The value of 0 indicates to use the default value of 4300 cylinder.
    • Max Cylinder no. for secondary quantity - Maximum number of cylinders that can be assigned to a secondary quantity. The value of 0 indicates to use the default value of 4300 cylinder.
    • Maximum size for disk data sets - Maximum number of cylinders (not for the data set) to be allocated on an alternative unit. If the primary quantity exceeds this dimension, the data set will be allocated on an alternative unit.
    • Alternative name for unit - Name of the alternative unit where you want to allocate the data set that exceeded the Maximum Size for Disk Data Sets value.
    • Primary quantity percentage - Percentage of cylinder you want to assign at the primary quantity in regard to the total dimension of output data set. The default value is 10%.
    • Secondary quantity percentage - Percentage of cylinder you want to assign at the secondary quantity in regard to the total dimension of primary quantity. The default value is 15%.
    • JCL output data set - Library where the generated JCLs will be stored.
    • JCL output member for deletion - F irst part of the name of the generated JCLs (the second part will be a 3-digit progressive number starting from 001).
    • JCL output member for allocation - First part of the name of the generated JCLs (the second part will be a 3-digit progressive number starting from 001).
    • Submit Job - Indicator of whether to submit the newly created JCLs.
Parent topic: Create unload jobs


JCL information

The JCLs are built starting from CBTUNLD1 skeleton, which can be customized. The CBTUNLD2 skeleton is an alternative skeleton where the UNLOAD utility is used instead of the DSNTIAUL utility.

The structure of the skeleton comprises a starting part and an ending part that are repeated once in the created JCLs, and a central part that is delimited by the following keywords:

//* SINGLE FILE ELABORATION and

//* SINGLE FILE END ELABORATION, that is repeated for each file.

In the skeleton some variables are used, and they are changed into the JCL values using the values stored in the Knowledge Base or included in the launch parameters.

The following table describes the used variables:

Variable | Description
&NUM | Numeric progressive
&UNLOUTNAM | Name of the sequential data set (output of extraction process)
&PCYL | Primary quantity
&LRECL | Record length
&UNIT | SYSDA or the value put in the launch screen, depending on the above described rules
&CYL | It can be TRK (if the space is less than a cylinder) or CYL
&SCYL | Secondary quantity
&OWNAME | Owner of the original table
&FILNAME | Name of the original table
&DBNAME | Database of the original table
&TSNAME | Tablespace of the original table
&UNLINPNAM | The unload name as it can be seen in "Work with files"
&STEP | The step name (automatically generated)
&JOBN | It is actualized with the JCL name
Parent topic: Create unload jobs


Create extraction jobs

The Create Extraction job executes a process to create a JCL that will execute an extraction method.
  • Job creation
  • JCL information
Parent topic: Data subset extraction jobs


Job creation

To create a Create Extraction job:

  1. From the Work with Jobs window, click New.
  2. In the List of Jobs section, select Create Extraction Jobs.
  3. In the General section, select the appropriate Machine ID and Company name.
  4. In the File Qualification section, specify values for the following fields:
    • Application ID - The application ID, if applicable.
    • DB2 Owner, DL/I DBD - The owner of the DB2 or DL/I DBD file where data will be masked.
    • File type - The type of file (ADA, VSAM, DB2, DL/I, GDG, SEQ, or ORA).
    • File name - The name of the file where data will be masked.
    • File version - The version number if the file is GDG.
    • Record format - Name of the record format.
  5. Click Apply to submit the job. The Secondary Options window appears.
  6. In the Secondary Options window, select the required Group and Method values, and specify values for the following fields:
    • From Step / To Step - Range of values for a subset of steps for the selected method.
    • Extraction Type:
      • Access Path Simulation - Indicator for the generated JCL to run the method in simulation access path.
      • Simulation - Indicator for the generated JCL to run the method in simulation, without populating the output files. (A first run in simulation is recommended in order to calculate spaces).
      • Real Extraction - Indicator to perform real extraction.
    • Discard Unchanged Files - Discard all DB2 tables from the allocation JCL that belong to the current method and are neither reduced nor masked.
    • Run Deletion - Indicator of whether to perform a deletion of all results of previous extractions for the same step range.
    • Run Extraction - Indicator of whether to perform the execution of the extractions for the selected step range.
    • JCL output data set - Library where the generated JCLs will be stored.
    • JCL output member - N ame of the generated JCLs.
    • Temporary data set prefix - Name of the prefix for the temporary data set used in the JCL in order to create the temporary unloads from image copy.
    • Submit Job - Indicator of whether to submit the newly created JCLs.
Parent topic: Create extraction jobs


JCL information

The JCLs are built starting from CBTESUB2 skeleton, which can be customized. The structure of the skeleton comprises a single part that is repeated once in the created JCLs, and is delimited by the following keywords:

//* UNLOAD ELABORATION and

//* UNLOAD END ELABORATION, by the keywords:

//* SINGLE FILE ELABORATION and

//* SINGLE FILE END ELABORATION and by the keywords:

//* DELETE ELABORATION and

//* DELETE END ELABORATION is repeated for each file.

In the skeleton some variables are used, and they are changed into the JCL values using the values stored in the Knowledge Base or included in the launch parameters.

The following table describes the used variables:

Variable | Description
&NAMEJOB | Name of the submitted job
&USER | User who is submitting the job
&URDB2 | DB2 subsystem where the product is installed
&URPLAN | DB2 plan of the product
&URLOAD | DB2 load library
&URRUNLIB | DB2 runlib library
&URSRUN | sceerun library
&URPROJ. | First qualifier of the product libraries
&URGRPER. | Second qualifier of the product customization libraries
&URGRPTF. | Second qualifier of the product PTF libraries
&URGRGROUP. | Second qualifier of the product libraries
&NAMEPGM | Name of the first program of the batch chain
&TNSRLNBR | Input parameter Machine
&TNCDSOC | Input parameter Company
&TNMETHOD | Input parameter Method
&TNGROUP | Input parameter Group
&TNSTEP | Input parameter From Step
&TNTOSTEP | Input parameter To Step
&TNSIMUL | Input parameter Simulation
&TNDISFIL | Input parameter Discard Unchanged Fields
&TNRUNDEL | Input parameter Run deletion
&TNRUNEXT | Input parameter Run extraction
&DDNAME | DD name automatically generated by the product
&FILNAME | Name of the unload file generated from image copy
&LABEL | Progressive number
&RECL | Record length for the unload file generated from image copy
&UROWNER | DB2 owner of the product tables
&URPROJ | First qualifier of the product libraries
&URGROUP | Second qualifier of the product libraries
&IMGCPY | Image copy data set
&TYPE | Unload file generated from image copy
&OWNAME | Owner of the original table
&TBNAME | Name of the original table
&DBNAME | Name of the database of the original table
&TSNAME | Name of the tablespace of the original table
&PQTY | Primary quantity
&CYL | TRK (if the space is less than a cylinder) or CYL
&SCYL | Secondary quantity
&URDPREFA | First part of the temporary data set names
&URDPREFB | Second part of the temporary data set names
&JOBN | It is actualized with the JCL name

Parent topic: Create extraction jobs


Create reload jobs

This function allows you to create a JCL to reload all DB2 tables with direct access included in a method or associated with a business protection (Masking) procedure.
  • Job creation
  • JCL information
Parent topic: Data subset extraction jobs


Job creation

To create a Create Reload job:

  1. From the Work with Jobs window, click New.
  2. In the List of Jobs section, select Create Reload Jobs.
  3. In the General section, select the appropriate Machine ID and Company name.
  4. In the File Qualification section, specify values for the following fields:
    • Application ID - The application ID, if applicable.
    • DB2 Owner, DL/I DBD - The owner of the DB2 or DL/I DBD file where data will be masked.
    • File type - The type of file (ADA, VSAM, DB2, DL/I, GDG, SEQ, or ORA).
    • File name - The name of the file where data will be masked.
    • File version - The version number if the file is GDG.
    • Record format - Name of the record format.
  5. Click Apply to submit the job. The Secondary Options window appears.
  6. In the Secondary Options window, select the required Group and Method values, and specify values for the following fields:
    • From Step / To Step - Range of values for a subset of steps for the selected method.
    • Max no. files in JCL - Maximum number of files you want process in a single JCL. If your method exceeds this limit, the function creates a new JCL for the other files. The default value is 2000.
    • Replace Data - Indicator of whether data replacement in the target database is enabled. If not selected, data items are added without deletion.
    • Create Syspunch - Indicator of whether to create the part of the JCL performing the SYSPUNCH .
    • Create Reload - Indicator of whether to create the part of the JCL performing the reload.
    • Target DB2 Owner - Name of the target DB2 owner. If this parameter is not specified, this value is the same as the source DB.
    • Target DSN - Name of the target database subsystem. If this parameter is not specified, this value is the same as the source DB.
    • JCL output data set - Library where the generated JCLs will be stored.
    • JCL output member for reload - Name of the generated JCLs used for reload.
    • JCL output member for repair - Name of the generated JCLs used for repair.
    • Submit Job - Indicator of whether to submit the newly created JCLs.
Parent topic: Create reload jobs


JCL information

The JCLs are built starting from CBTELOD1 and CBTELOD2 skeletons, which can be customized. The CBTUNLD2 skeleton is an alternative skeleton where the UNLOAD utility is used instead of the DSNTIAUL utility in order to create SYSPUNCHs .

The structure of the skeleton comprises a starting and ending part that are repeated once in the created JCLs, and a central part that is delimited by the following keywords:

//* SINGLE FILE ELABORATION and

//* SINGLE FILE END ELABORATION, that is repeated for each file.

In the skeleton some variables are used, and they are changed into the JCL values using the values stored in the Knowledge Base or included in the launch parameters.

The following table describes the used variables:

Variable | Description
&NUM | Numeric progressive
&UNLOUTNAM | Name of the sequential data set (output of extraction process)
&OWNAME | Owner of the original table
&FILNAME | Name of the original table
&DBNAME | Database name of the original table
&TSNAME | Tablespace name of the original table
&UNLINPNAM | Name of the unload as specified in the Work with files area
&STEP | Step name for the SYSPUNCH generation (automatically generated)
&STP1 | Step name for the MISYSPUN program call (automatically generated)
&STP2 | Name of the reload step (automatically generated)
&SYSPARM | It can be ADD,YES, number of sortkeys if no replace parameter, REP,YES, number of sortkeys if replace parameter
&DSNNAME | Name of the input DSN
&METHOD | Name of the method
&JOBN | Name of the job actualized with the JCL name
Parent topic: Create reload jobs


Available operations

The buttons at the bottom of the Secondary Options window are:

  • Reset All - Resets all parameters to their default values.
  • OK - Saves any changes that have not been applied and closes the window.
  • Cancel - Cancels any changes made to the options. Any actions confirmed with the Apply button cannot be restored with Cancel but must be entered and confirmed again.
Parent topic: Data subset extraction jobs


Distributed Exporter

The Distributed Exporter is a utility that creates and provides necessary information to the Extension Technology so that actual subsetting of your distributed data store can occur.
This utility generates the following files after you have successfully exported from the Data Subset Extraction module:
  • method.rc - Coded content of source and target databases information.
  • method.txt - Information for the environment and the method in it which are about to be masked.
  • filter.txt - All created filters in the test environment.
  • elab.txt - Data that is about to be masked.
  • cbfld.txt - Information about combined fields used in the method.
  • CREATETABLE.sql - list of the tables that need to be masked; a generic CREATE TABLE statement is provided for each table.
  • CREATEINDEX.sql - list of all the indexes for the tables that need to be masked; a generic CREATE INDEX statement is provided for each index.
  • ALTERTABLE_RI.sql - list of all the primary and foreign keys for the tables that need to be masked; a generic ALTER TABLE statement is provided for appropriate primary and foreign keys.
There are two areas of interest when it comes to using the Distributed Exporter with distributed data stores.

  • If you use ODBC-enabled data stores, you can subset across all your data stores with one invocation of the ODBC Extension. This feature is not available with the Oracle Extension.

  • You also have the ability to subset within one distributed data store, as long as a distinct target schema name is provided.

Note: If you use the Distributed Exporter utility on one machine, and do your subsetting on another, make sure that the target ODBC DSN matches the ODBC DSN used for the actual subsetting. Similarly, if you use the Oracle Extension, make sure your Oracle TNS service names match.

To access the Distributed Exporter window, click [image] from the main window.

  • Window contents
  • Available operations
Parent topic: Data Subset Extraction Guide


Window contents

This section describes the fields listed in the Distributed Exporter window.

To export a method from the Data Subset Extraction module, you need to provide the following input:

  • Your workspace (machine ID, company).
  • Your source data store credentials (source database, source schema, source user ID/password).
    • For a data store that you will access using the ODBC Extension, the source database is an ODBC DSN configured to access that data store.
    • For a data store that you will access using the Oracle Extension, the source database is a BDE alias configured to access that data store. For more information, see the Oracle Alias Creation section in Getting Started with Distributed Data Stores.
  • Your target data store credentials (target database, target scheme, target user ID/password).
    • For a data store that you will access using the ODBC Extension, the target database is an ODBC DSN configured to access that data store.
    • For a data store that you will access using the Oracle Extension, the target database is an Oracle TNS service name configured to access that data store. For more information, see the Oracle Alias Creation section in Getting Started with Distributed Data Stores.
  • Directory where you want to place output files for Extension Technology (Output directory).
Note: If the source database and the target database are the same, the schemas cannot be the same.

In addition, if you are using ODBC-enabled data stores, you can provide multiple sets of source and target data store credentials.

Parent topic: Distributed Exporter


Available operations

The buttons at the bottom of the Distributed Exporter window are:

  • Export - Starts the export procedure, which generates the files.
  • Close - Closes the window.
Parent topic: Distributed Exporter


Distributed Statistics Loader

For the Data Express for Distributed Data Stores solution, the Distributed Statistics Loader is a utility that lets you load run-time statistics into the KB. Once you have executed the Extension Technology to subset your data, a log filed called dxe_statistics.txt is created. This file is used to load the run-time data into the KB so that the statistics are available in Data Subset Extraction.

For more information on using the Distributed Statistics Loader, see Extension Technology Utilities in Getting Started with Distributed Data Stores.

To access the Distributed Statistics Loader window, click [image] from the main window .

  • Window contents
  • Available operations
Parent topic: Data Subset Extraction Guide


Window contents

This section describes the fields listed in the Distributed Statistics Loader window as shown below:

Figure 1. Figure 16-1: Distributed Statistic Loader

To use the Distributed Statistics Loader:

  1. In the Log file to import box, specify the dxe_statistcs.txt log file name including the directory path where you saved it on your Windows machine.
  2. Click Check Log File to verify that the log file you select is correct.

    The remaining fields in the Distributed Statistic Loader windows populate from information specified from the dxe_statistcs.txt file.

  3. Click Load to load the subset extraction statistics into Data Subset Extraction.

    A Statistic loaded successfully! message is displayed.

    You can view the run-time statistics in various places in Data Subset Extraction. For example:
    • Viewing the List of Elaborations in Step by selecting the appropriate method from the Main Window
    • Viewing the Data Stores Elaboration from the Elaborations tab in the Work with Method window
    • Viewing the List of Elaborations in Step from the Method Overview tab in the Work with Method window

Run-time statistics include the following:

  • Number of Read Records
  • Number of Written Records
  • Reduction in Percentage
  • Simulation Date
  • Real Extraction Date
  • Elapsed Time [hh:mm:ss]
  • CPU Time [hh:mm:ss]
Note: The Reduction in Bytes run-time statistic is not used in the Distributed Systems solution because of the way ODBC-enabled data stores display the record length.
Parent topic: Distributed Statistics Loader


Available operations

The buttons at the bottom of the Distributed Statistics Loader window are:

  • Check Log File - Verifies that the log file you selected is correct.
  • Cancel - Closes the window.
  • Load - Loads the subset extraction statistics into Data Subset Extraction.
Parent topic: Distributed Statistics Loader


Work with Method multiple filters

In order to create multiple filters adding conditions like "OR" or "AND" to a sub-extraction query, you can apply a new class with a filter to a second column of a data store already included in any method or you can create a combined data element with fields you need to use and include it in a method.
  • Introduction to Work with Method multiple filters
  • More than one single filter for the same data store
  • More than one combined filter involving more than one data element
Parent topic: Data Subset Extraction Guide


Introduction to Work with Method multiple filters

When more than one filter (single or combined) is added to the same data store, the sub-extraction query is created with UNION or INTERSECT SQL statements. This enables extraction of records that satisfy the filter criteria.

To better describe the two different situations, this guide divides the explanation in two different case:

  • More than one single filter for the same data store. This filter adds an "OR" or an "AND" condition to the query.
  • More than one combined filter for the same data store, involving more than one data element. This filter adds an "OR" or an "AND" condition to the query

For both cases, we used a DB2 table named CUSTOMER, with columns NAME and SURNAME and related class NAME and SURNAME.

Parent topic: Work with Method multiple filters


More than one single filter for the same data store

This following steps describe how to apply the second filtered class SURNAME to the second column SURNAME of a data store CUSTOMER which has been included in a Method from the Work with Method window. The query is created with a UNION statement within the two filter criteria.

At the end of the Method confirmation phase, in order to apply the second class, copy the data store to a new step by clicking Copy Selected Step. To copy the data store:

  1. Select the appropriate row in the Work with Method window and click the button.
  2. Right-click on new row to display the context menu, point to Elaboration, and then click Clear Selection class and Disconnect Filter.
  3. Right-click on new row to display the context menu, point to Elaboration, and then click Selection class / Filter properties and select the class SURNAME assigned to column SURNAME.
  4. Right-click on new row to display the context menu, point to Move selected to and then click 20 - STEP CLASSES.

To create a query with the INTERSECT statement, follow steps 1 through 4 above, and then:

  1. Select the two rows related to the DB2 table CUSTOMER.
  2. Right-click; then select Utility from the context menu.
  3. Select the Create filter in AND voice.
  • Filter types
  • Filter examples more than one single filter
Parent topic: Work with Method multiple filters


Filter types

This section describes the filter types that can be applied to the primary and secondary classes listed in the List of class in the elaboration grid in the Selection class / Filter properties window.

If you want to apply a filter to a primary and secondary class, you must select each class from the list and later select the type of filter you want to specify. This operation must be repeated for the secondary class in the list for which you want to apply a filter.

Note: The filter values for date fields must be in the format YYYYMMDD or YYYYMMDD HHHH.MM.SS where HHHH is based on a 24-hour clock.
Note: After associating the correct filter with each class, click OK.
Parent topic: More than one single filter for the same data store


Filter examples more than one single filter

The following examples show various ways to code for more than one single filter:

Example: Filter by range in OR (UNION)
The following example reproduces the query created using filter by range value for NAME and SURNAME , where the values are:
  • from ALBERT to STEPHEN for NAME
  • from LENNON to MORANIS for SURNAME
SELECT A.* FROM "DELGT"."CUSTOMER" A
WHERE A.NAME BETWEEN 'ALBERT' AND 'STEPHEN'
UNION
SELECT A.* FROM DELGT.CUSTOMER A
WHERE A.SURNAME BETWEEN 'LENNON' AND 'MORANIS'
Example: Filter by range in AND (INTERSECT)
The following example reproduces the query created using filter by range value for NAME and SURNAME , where the values are:
  • from ALBERT to STEPHEN for NAME
  • from LENNON to MORANIS for SURNAME
SELECT A.* FROM "DELGT"."CUSTOMER" A
WHERE A.NAME BETWEEN 'ALBERT' AND 'STEPHEN'
INTERSECT
SELECT A.* FROM DELGT.CUSTOMER A
WHERE A.SURNAME BETWEEN 'LENNON' AND 'MORANIS'
Example: Filter by list value in OR (UNION)
The following example reproduces the query created using filter by list value for NAME and SURNAME , where the values are:
  • ALBERT, TOM and STEPHEN for NAME
  • LENNON, ROSS and MORANIS for SURNAME
SELECT A.* FROM "DELGT"."CUSTOMER" A
WHERE ((A.NAME) IN ('ALBERT','TOM' ,'STEPHEN'))
UNION
SELECT A.* FROM DELGT.CUSTOMER A
WHERE ((A.SURNAME) IN ('LENNON', 'ROSS', 'MORANIS'))
Example: Filter by list value in AND (INTERSECT)
The following example reproduces the query created using filter by list value for NAME and SURNAME , where the values are:
  • ALBERT, TOM and STEPHEN for NAME
  • LENNON, ROSS and MORANIS for SURNAME
SELECT A.* FROM "DELGT"."CUSTOMER" A
WHERE ((A.NAME) IN ('ALBERT','TOM' ,'STEPHEN'))
INTERSECT
SELECT A.* FROM DELGT.CUSTOMER A
WHERE ((A.SURNAME) IN ('LENNON', 'ROSS', 'MORANIS'))
Parent topic: More than one single filter for the same data store


More than one combined filter involving more than one data element

This following section describes how to apply the filtered class CLA0202 assigned to the combined data element which has been created including the column NAME and SURNAME of data store CUSTOMER, from the Work with Method window.

  • Filter types
  • Filter examples more than one data element
Parent topic: Work with Method multiple filters


Filter types

This section describes the filter types that can be applied to the primary classes listed in the List of class in elaboration grid in the Selection class / Filter properties window.

If you want to apply a filter to a class and there are several primary selection classes, you must select each class from the list and later select the type of filter you want to specify. This operation must be repeated for each class in the list for which you want to apply a filter.

Note: The filter values for date fields must be in the format YYYYMMDD or YYYYMMDD HHHH.MM.SS where HHHH is based on a 24-hour clock.
Note: After associating the correct filter with each class, click OK.
Parent topic: More than one combined filter involving more than one data element


Filter examples more than one data element

Example: Filter by range

The following example reproduces the query created using filter by list value for NAME and SURNAME, where the values are:

  • ALBERT,TOM and STEPHEN for NAME

  • LENNON, ROSS and MORANIS for SURNAME

SELECT * FROM "DELGT"."CUSTOMER" A, OWUR41SV.HSENVPAR B WHERE B.METHOD = 'DOUBCOND3 ' AND B.MCRECID = 0028 AND B.CLATYPE = 0202 AND B.FLTPRG = 0001 AND A."NAME" = ( ( SUBSTR(B.VALUE, 001, 040) ) ) AND A."SURNAME" = ( ( SUBSTR(B.VALUE, 041, 040) ) ) AND B.PARMPRG > 2

Example: Filter by list value

The following example reproduces the query created using filter by list value for NAME and SURNAME, where the values are:

  • ALBERT,TOM and STEPHEN for NAME
  • LENNON, ROSS and MORANIS for SURNAME
SELECT * FROM "DELGT"."CUSTOMER" A, OWUR41SV.HSENVPAR B WHERE B.METHOD = 'DOUBCOND3 ' AND B.MCRECID = 0028 AND B.CLATYPE = 0202 AND B.FLTPRG = 0001 AND A."NAME" = ( ( SUBSTR(B.VALUE, 001, 040) ) ) AND A."SURNAME" = ( ( SUBSTR(B.VALUE, 041, 040) ) ) AND B.PARMPRG > 2

Example: Filter by filter list

. The following example reproduces the query created using filter by filtered where class NAME and class SURNAME have been previously created like output class for another entry within the same method:

SELECT * FROM "DELGT"."CUSTOMER" A, OWUR41SV.ANENVLST B WHERE B.METHOD = 'DOUBCOND5 ' AND B.MCRECID = 0028 AND B.CLATYPE = 0202 AND B.FLTPRG = 0001 AND A."NAME" = ( ( SUBSTR(B.VALUE, 001, 040) ) ) AND A."SURNAME" = ( ( SUBSTR(B.VALUE, 041, 040) ) )
Parent topic: More than one combined filter involving more than one data element


Error Messages

Provides a detailed list of all Data Express warning, error, and informational messages displayed in log files during batch processing or on the screen during elaboration processing.

This document is addressed to all users who use the Data Express for z/OS solution or for users of either the Data Express for z/OS or Data Express for Distributed Systems solution who have made use of the Extension Technology.

  • Error Message Format
    Describes the general format of Data Express mainframe, extension, and Oracle/ODBC messages.
  • Mainframe Messages WRC0001 to WRC0199
    Details Data Express mainframe messages in the specified message ID range.
  • Mainframe Messages WRC0200 to WRC0299
    Details Data Express mainframe messages in the specified message ID range.
  • Mainframe Messages WRC0300 to WRC0399
    Details Data Express mainframe messages in the specified message ID range.
  • Mainframe Messages WRC0400 to WRC0499
    Details Data Express mainframe messages in the specified message ID range.
  • Mainframe Messages WRC0500 to WRC0599
    Details Data Express mainframe messages in the specified message ID range.
  • Mainframe Messages WRC2000 to WRC4999
    Details Data Express mainframe messages in the specified message ID range.
  • Data Express Extension Messages
    Details Data Express extension messages.
Parent topic: Data Express


Error Message Format

Describes the general format of Data Express mainframe, extension, and Oracle/ODBC messages.

The format for error messages is as follows:

Type | Format | Description
MVS WRCxxxx:message
  • WRC is the mainframe identifier.
  • xxxx is the error message number.
  • message is the message text.
Data Express Extension DE-xxxx:message
  • DE is the extension identifier.
  • xxxx is the error message number.
  • message is the message text.
Oracle and ODBC | SQLCODE=xxxxx,SQLSTATE=yyyyy: message | xxxxx, yyyyy and message are retrieved directly from the ODBC driver/data store.
Parent topic: Error Messages


Mainframe Messages WRC0001 to WRC0199

Details Data Express mainframe messages in the specified message ID range.
Message ID | Message Text | Description | Resolution | Phase of Reference
Error Messages | No data to list. | The selection contains no data. |  |
WRC0029 | Unexpected SQL error in program ProgramName, table TableName, operation OperationName, SQLCODE SQLCode. | An error has occurred in an SQL operation during the launched batch operation. | Verify the availability of the specified table. If the error persists, contact Micro Focus Product Support for technical assistance. |
WRC0031 | The software key is invalid or is not enabled. | Data Express has not been enabled correctly. | Make sure that you have entered the correct password. If the problem persists, contact Micro Focus Product Support for password verification or the release of a new license. |
WRC0033 | The maximum number of files allowed by the license has been reached. | The maximum number of files allowed by the Data Express license code owned by the user has run out. | Contact Micro Focus Product Support for password verification or the release of a new license. |
WRC0045 | Multiple selection is not allowed. | A multiple selection operation cannot be performed. |  |
WRC0046 | Name contains invalid characters. | During the Toolkit Menu editing, there are one or more invalid characters. | Verify the invalid character and delete it. |
WRC0092 | Error in opening file FileName - File status: FileStatus. | An error has occurred during the processing of the specified file. | Make sure that accessing the file is possible. If the error persists, contact Micro Focus Product Support for technical assistance. |
WRC0093 | Read/write error in file FileName - database code: DatabaseCode. | An error has occurred during the processing of the specified file. | Check whether you can access the file. If the error persists, contact Micro Focus Product Support for technical assistance. |
WRC0110 | Required field. | During the Toolkit Menu mask editing, there are one or more empty fields. | Verify the empty field and fill it. | Toolkit's programs.
WRC0125 | A field contains invalid characters. | During the editing of the "Submit Client Scheduled Job" mask from the toolkit, there are one or more empty fields. | Verify the empty field and fill it. | Toolkit's programs.
WRC0186 | The process identifier ProcessID was not found. | The specified process identifier does not exist in the system. | Make sure that you have entered the correct process identifier name. If necessary, verify the presence in the system of the process identifier you want to use in the Work with Process IDs window. |
WRC0188 | The file name FileName is not a valid GDG name. | The name entered for the GDG file is not correct. | Make sure the file has the correct syntax, for example, that it includes the version number. |
WRC0190 | The company MachineID - CompanyName does not exist. | The value entered into the Company field does not exist in the system for the specified Machine ID. | Make sure that you have entered the correct company name. If necessary, verify the presence in the system of the company you want to use with the Work with company window. |
WRC0191 | File FileName has already been loaded into the data repository. | The specified file has already been loaded in the repository; that is, a file with that name and of that type exists in the product. | If you want to load a different version of the file, you must first remove the previous version of the file by clicking Delete in the Work with file window. | File loading
WRC0192 | More than 1000 formats were found for file FileName. | More than 1000 record formats for the specified file have been identified. The product cannot process multi format files with more than 1000 record formats. |  | File loading
WRC0193 | No records were found for file FileName. Formats cannot be loaded. | No records for the specified file have been identified. Formats cannot be loaded. | Check whether the file is only temporarily empty. Repeat the loading of the file when the file contains some records. | File loading
WRC0194 | The application identifier ApplicationID was not found. | The specified application code does not exist in the system. | Make sure you have entered the correct application code name. If necessary, verify the presence in the system of the application code you want to use with the Work with Application IDs window. |
WRC0195 | The copybook name CopybookName is invalid. | The name of the SYSPUNCH or DCLGEN does not have the correct format during the loading of DB2 files from the sequential unload. The correct formats are dsname(srcname) or dsname. |  | DB2 analysis
WRC0197 | x sources have been processed, and x sources have been loaded into the data repository. | The message is a warning about the result of the processing job. |  | Copybook loading
WRC0198 | Check the process identifier parameters. More information is requested for elaboration. | The process identifier indicated for the processing of the DL/I file does not contain all the information required to load the DL/I file correctly. | Use the Work with Process IDs window to make sure the process identifier you want to use contains all the information required to process the DL/I file typology. | File Loading
Parent topic: Error Messages


Mainframe Messages WRC0200 to WRC0299

Enter the syntax information of your reference here (optional).
Message ID | Message Text | Description | Resolution | Phase of Reference
WRC0201 | The field must be set to space. | During the mask editing of "Creation Delete Define Sources" Job from toolkit, there are one or more full fields. | Verify the full field and add spaces to it. | Toolkit's programs
WRC0202 | There is not enough information in the process identifier to retrieve the segment record lengths. | The process identifier found that the DL/I file does not contain all the information required to load the DL/I file correctly. | Use the Work with Process IDs window to make sure the process identifier you want to use contains all the information required to process the DL/I file typology. | File loading
WRC0203 | An error occurred during the analysis of SYSPUNCH SYSPUNCH-Name. | An error during the analysis of the specified SYSPUNCH has occurred. | Check the SYSPUNCH source to ensure the correct SYSPUNCH is entered for the DB2 table you want to load. If the error persists, contact Micro Focus Product Support for technical assistance. | File loading
WRC0205 | The INTO clause was not found in SYSPUNCH SYSPUNCH-Name. | The INTO clause for the specified SYSPUNCH cannot be found during the end of the loading phase for a DB2 table through an access to the sequential unload. | Check the SYSPUNCH source to verify the lack of the INTO clause. If necessary, contact Micro Focus Product Support for technical assistance. | File loading
WRC0206 | A variable length field was found as the last column for SYSPUNCH SYSPUNCH-Name. Check the table record length. | A VARCHAR in the last column for the specified SYSPUNCH has been identified at the end of the loading phase of a DB2 file through an access to the sequential unload. This field will be loaded with a length of 256. |  | File Loading
WRC0207 | More than 1500 columns were found for SYSPUNCH SYSPUNCH-Name. | More than 1500 columns are identified for the specified SYSPUNCH at the end of the DB2 table loading phase through an access to the sequential unload. A SYSPUNCH exceeding 1500 columns cannot be processed. |  | File loading
WRC0213 | An error occurred during the analysis of copybook CopybookName. | A problem with the search for keywords during the analysis of copybook/include statements has occurred. | If they do not, contact Micro Focus Product Support for technical assistance. |
WRC0214 | A nested copybook was found in CopybookName. Data Express is not able to process nested copybooks. | Nested copybooks (or copybooks that cannot be processed) have been identified. | Nested copybooks are not generally used to describe the layout of a file, so the message can likely be ignored. Should the copybook describe a file layout, it would be necessary to turn the nested copybook into a simple copybook. | Copybook, include analysis
WRC0215 | More than 32767 rows were found in copybook CopybookName. The analysis will be terminated. | The presence of more than 32767 rows for the specified copybook have been identified. The product cannot process copybooks containing such a large number of rows. | This type of copybook is not generally used to describe the layout of a file, so the message can likely be ignored. Should the copybook describe a file layout, it would be necessary to split up the copybook. | Copybook, include analysis
WRC0216 | The copybook name CopybookName is not valid as a file description. | The message is a warning explaining that the analyzed copybook is not compatible with the description of a file layout. To show the description of a file, the copybook must be a working copybook; that is, one that contains definitions of fields. |  | Copybook analysis
WRC0217 | No word could be found in string StringName . | Unexpected syntax in the copybook has been identified. This does not normally occur with copybooks describing file layouts. | If the copybook describes a file layout, make sure the generated fields correspond to the actual conditions of the file. If the generated fields do not correspond to the true conditions of the file, the recommended workaround is the creation of a simplified copy of the copybook with field statements only. | Copybook, include analysis
WRC0218 | The first word of string StringName is longer than expected. | Unexpected syntax in the copybook has been identified. This does not normally occur with copybooks describing file layouts. | If the copybook describes a file layout, make sure the generated fields correspond to the actual conditions of the file. If the generated fields do not correspond to the true conditions of the file, the recommended workaround is the creation of a simplified copy of the copybook with field statements only. | Copybook, include analysis
WRC0219 | Internal error processing string StringName . | A general problem has been found with the search for keywords within the copybook/include analysis. | Make sure the generated fields correspond to the actual conditions of the file. Otherwise, contact Micro Focus Product Support for technical assistance. | Copybook, include analysis
WRC0220 | A length greater than 32767 was found for an elementary field in copybook StringName . The length 32767 will be assumed. | An elementary field longer than 32767 bytes for the specified copybook has been identified. As a result, the product will store a 32767-byte long field ignoring the rest. | This message can be ignored. | Copybook, include analysis
WRC0221 | Level 66 has been found in copybook CopybookName and will be ignored. | A 66 level field in the specified copybook has been identified. These fields will be ignored. | This message can be ignored. | Copybook, include analysis
WRC0222 | More than 99999 fields have been found in copybook CopybookName. Fields beyond 99999 will be ignored. | More than 99999 fields in have been identified. As a result, the product will store the first 99999 fields and ignore all the others. | Make sure the generated fields correspond to the effective conditions of the file. | Copybook, include analysis
WRC0223 | An undefined field FieldName was found in copybook CopybookName. | The definition of the field referenced in the specified copybook has not been found. This generally occurs when the REDEFINES clause references an invalid field. | Make sure the generated fields correspond to the actual conditions of the file. | Copybook, include analysis
WRC0224 | Multiple occurrences of field FieldName have been found in copybook CopybookName. | A field has been defined several times in the specified copybook. This generally happens as a result of an unexpected COBOL syntax. | Make sure the generated fields correspond to the actual conditions of the file. | Copybook, include analysis
WRC0225 | Field FieldName of I/O area I/O-Area-Name could not be divided because the I/O area contains the OCCURS clause. | The product cannot divide a field of the specified I/O area because the I/O area contains the OCCURS clause. | Make sure the generated fields correspond to the actual conditions of the file. | Copybook, include analysis
WRC0226 | The maximum number of incongruent formats for I/O area I/O-Area-Name has been reached. Field definition splitting is skipped. | While analyzing the specified I/O area, a number of incongruent formats exceeding the specified value Maximum Progressive of I/O Area have been identified. As a result, separate I/O areas for each incongruent format are not created. This happens when an I/O Area with several REDEFINES clauses is found. |  | Copybook, include analysis
WRC0227 | No copybook was found to analyze specified parameters. | No copybook has been found for the specified launch parameters. | Make sure the copybooks you want to analyze have been loaded into the environment with the Work with Copybook function and that the launch parameters specified for the copybook analysis are correct. | Copybook analysis phase
WRC0228 | Error found in I/O operation on data set Data-Set-Name - File status: FileStatus. | An error in an I/O operation has occurred during batch processing. | Verify the availability of the specified data set. If the error persists, contact Micro Focus Product Support for technical assistance. |
WRC0230 | Direct access to the DL/I file is not allowed for file analysis. | Direct access for file analysis was attempted, but is not allowed for the loading and analysis phases. | Modify the launch parameter and relaunch the file analysis. | File loading
WRC0231 | More than 99 DL/I segments were found. Additional segments will not be handled. | More than 99 segments for the DL/I file have been found. The product does not handle DL/I files with such a large number of segments. |  | DL/I analysis
WRC0240 | The file type FileType is unknown. | An error message occurred as a result of a data check on the data entered. The specified file type is unknown to the system. | Make sure you have entered the correct file typology. Correct values for file type are described in the Data Model Guide. |
WRC0242 | The file name Filename is not correct for file type FileType. | The file name has not been correctly specified. | Make sure the file used for the import contains the values required for each file type. In particular, make sure the FILENAME field contains the file name for DB2 files, the segment name for DL/I files, and the data set name for GDG or VSAM sequential files. Then, relaunch the import phase. Correct syntax for file name is described in the Data Model Guide. | File loading
WRC0247 | No tables found with matching input criteria. | The data store name included in the Input file for the execution of "Load Sequential File" Job is invalid. | Verify the correct name of the data store. | Load Sequential File from toolkit
WRC0248 | The table TableName cannot be accessed. SQLCODE SQL-Code. | Direct access to a table was attempted but refused due to the table being locked. | Check whether there are jobs locking the table. |
WRC0249 | Burldb4 - unrecognized, SQL type: SQL-Type. | A problem has been found with the direct access to the table due to the presence of an unrecognized type of column. | If necessary, contact Micro Focus Product Support for technical assistance. |
WRC0259 | Found and discarded preprocessor statement PreprocessorStatement. | Statements have been found that are related to preprocessors inside the copybook. These instructions are ignored. |  | Copybook analysis
WRC0260 | Error found processing source SourceName. | A general message has been generated due to a problem with the search for keywords within the copybook analysis. This does not generally occur with copybooks. | Make sure the generated fields correspond to the actual conditions of the file. Otherwise, contact Micro Focus Product Support for technical assistance. | Include analysis
WRC0261 | Error found in analysis of the include IncludeName. | A general message has been generated due to a problem with the search for keywords within the copybook analysis. This does not generally occur with copybooks. | Make sure the generated fields correspond to the actual conditions of the file. Otherwise, contact Micro Focus Product Support for technical assistance. | Include analysis
WRC0262 | A nested include in IncludeName was found and discarded. | A nested include (which is a type of include that cannot be processed) has been found. | Nested includes are not generally used to describe a file layout so the message can be ignored. Should the include describe a file layout, it would be necessary to turn a nested include into a simple include. | Include analysis
WRC0263 | More than 32767 rows were found in include IncludeName. The analysis will be terminated. | More than 32767 rows for the specified include have been found, and cannot process the sources. | This message can be ignored. Should the include describe a file layout, it would be necessary to split the include up. | Include analysis
WRC0264 | The include IncludeName is not valid for a file description. | The message is a warning explaining that the analyzed include is not compatible with the description of a file layout. To show the description of a file, the include must contain field definitions. |  | Include analysis
WRC0265 | Data items with external attributes have been found in include IncludeName. | External attributes (PL/I concept) that are not handled have been found. This does not generally occur with file includes. | Make sure the generated fields correspond to the actual conditions of the file. Otherwise, contact Micro Focus Product Support for technical assistance. | Include analysis
WRC0266 | Nested factoring in include IncludeName was found and discarded. | Nested factoring (PL/I concept) that is not handled has been found. This does not generally occur with copybooks. | Make sure the generated fields correspond to the actual conditions of the file. Otherwise, contact Micro Focus Product Support for technical assistance. | Include analysis
WRC0270 | The DBD direct-access request was discarded. Processing for DatabaseName must be through sequential unload. | An error has occurred as a result of the launch of phases for DL/I files. The direct access is set for these files, but these phases only support the processing of DL/I files through sequential unload. The product will not process these DL/I files. | Change the process identifier as direct access is not supported for DL/I files. |
WRC0277 | Unexpected syntax was found in statement StatementName of include IncludeName. | Unexpected syntax in the specified include has been found. | Should the copybook/include describe a file layout, make sure the generated fields correspond to the actual conditions of the file. If the generated fields do not correspond to the actual conditions of the file, the recommended workaround is creating a simplified copy of the copybook/include with field statements only. |
WRC0278 | A defined data item was found inside a structure in include IncludeName. | Unexpected syntax in the specified include has been found. | Make sure the generated fields correspond to the actual conditions of the file. After an incorrect generation of fields, the recommended workaround is creating a simplified copy of the include with field statements only. | Include analysis
WRC0279 | An unhandled data item or token was found in include IncludeName at line LineNumber. | Unexpected syntax in the specified include has been found. | Make sure the generated fields correspond to the actual conditions of the file. After an incorrect generation of fields, the recommended workaround is creating a simplified copy of the include with field statements only. | Include analysis
WRC0280 | x OCCURS clauses were found for field FileName in copybook CopybookName. x OCCURS clauses assumed. | During analysis of the specified copybook, the number of OCCURS clauses in fields exceeded the limit Data Express can handle. Therefore, a set number of OCCURS clauses are assumed and the remaining ones have been ignored. The maximum number of handled OCCURS clauses is 256. |  | Copybook, include analysis
WRC0281 | Too many end statements were found in include IncludeName. | An unexpected END instruction has been found. | This message can be ignored. | Include analysis
WRC0282 | An error occurred during the parsing of include IncludeName. | A nested include (an include inside another include) with some unrecognized syntax has been found. | Make sure the generated fields correspond to the actual conditions of the file. Otherwise, contact Micro Focus Product Support for technical assistance. | Include analysis
WRC0283 | x OCCURS clauses were found for field FieldName in include IncludeName. x OCCURS clauses assumed. | During the analysis of the specified include, the product has found a number of OCCURS clauses that exceeds the limit handled by the product. Therefore, a number of OCCURS clauses have been assumed and the remaining ones have been ignored The maximum number of handled OCCURS clauses is 256. |  | Copybook, include analysis
WRC0286 | The I/O area I/O-AreaName has already been defined in a procedure in include IncludeName.The request was discarded. | Unexpected syntax in the specified include has been found. | Make sure the generated fields correspond to the actual conditions of the file. After an incorrect generation of fields, the recommended workaround is creating a simplified copy of the include with field statements only. | Include analysis
WRC0287 | The open I/O area I/O-Area-Name was found in include IncludeName. | Unexpected syntax in the specified include has been found. | Make sure the generated fields correspond to the actual conditions of the file. . In case incorrect fields are generated, the recommended workaround is creating a simplified copy of the include with field statements only. | Include analysis
WRC0288 | The Copybook-Library-Name, CopybookName was discarded. | An error message occurred at the end of the copybook loading phase (JCL statement containing the VSAM file definitions) because the specified copybook/source of the indicated type has not been loaded in the product repository as it was not recognized. This message is generated only if the Load only Handled Languages option is selected when the phase is launched. In general, unrecognized copybooks or sources are discarded because they are not useful to the processing operations. |  | Force recognition, copybook loading
WRC0289 | The copybook CopybookName was already loaded in the data repository. The request was discarded. | The specified copybook has already been loaded in the product repository. | If you want to load a different version of the copybook, remove first the previous version of the file by selecting the Delete function in the Work with Copybooks window. | Copybook loading
WRC0294 | Decimal columns were found for table TableName. Check the number of integer and decimal digits. | A message has occurred during the DB2 table loading phase through sequential unload with specification of the SYSPUNCH, which explains that columns have been defined as DECIMAL. Information on the number of integers and decimals in these columns from the SYSYPUNCH cannot be retrieved, so all digits specified in the columns are loaded as integers. | If the DECIMAL columns do not contain decimals, the message can be ignored. Otherwise, to display fields with the correct separation between integer and decimal digits, you must load the table again replacing the SYSPUNCH with the table DCLGEN. | Table loading
Parent topic: Error Messages


Mainframe Messages WRC0300 to WRC0399

Details Data Express mainframe messages in the specified message ID range.
Message ID | Message Text | Description | Resolution | Phase of Reference
WRC0308 | The monitoring rule is not allowed for numeric and alphabetical classes. | This is a result of a check on the entered data within the Work with classes function. The product does not allow monitoring rules to be defined at class level for classes that can be associated with either numeric or alphanumeric fields. |  |
WRC0319 | No file was found with the specified parameters. | A batch function has been launched by specifying, for the launch parameters, values that have no correspondence with a file loaded in the environment. | Make sure the files you want to process have been loaded with the Work with files function and that the launch parameters specified for the processing are correct. |
WRC0321 | The data set or member is empty. | The message displayed is a result of a check on the entered data. |  |
WRC0325 | Number of steps required. | The field From Step or To Step of the Deletion Job Generation mask contains non-numeric characters. | Insert numeric characters into the field. | Deletion Job Generation
WRC0326 | From Step cannot be greater than To Step. | The field From Step cannot contain a number greater than that contained in the field To Step. | Correct the number inserted into the From Step field. | Deletion Job Generation
WRC0327 | Value ValueName was not inserted in the list for method MethodName, step StepNumber , class ClassName. | This is a result of a check on the entered data. For DL/I, during each processing operation, an output is produced in the list between a parent segment and the following segment only for the first 9999 separate values. |  | Test environment generation
WRC0328 | File FileName was not found for machine ID MachineID, company CompanyName. | A message has been generated in the error log at the end of the test environment generation phase. The product has highlighted that the specified file, which has been subjected to extraction, is missing from the repository. The file is likely to have been removed from the environment. |  | Test environment generation
WRC0330 | Unable to retrieve Entity information for file FileName. | The message displayed is a result of a check on the entered data for the file deleting function. The message prevents files for which there are balancing rules or test environment extraction methods from being deleted from the scheduler. |  |
WRC0332 | Direct access for the file FileName is not allowed for data masking routines. | The Data Masking batch phase has not been performed because it had been launched for a DL/I file with direct access. The Data Masking phase can be performed for a DL/I file only on sequential unload. |  | Data Masking
WRC0333 | Direct access for DL/I files is not allowed for environment creation. | The extraction batch phase has not been performed because it had been launched for a DL/I file with direct access. The Environment Creation phase can be performed for a DL/I file only on sequential unload. |  | Environment creation
WRC0338 | Specify method or group. | The field Method or group cannot contain spaces. | Insert the method name or the group name. | Deletion Job Generation/ Creation Delete Define Sources
WRC0341 | No parent segment was found for DL/I DBD DatabaseName. | Extraction has not been performed because the name of a parent segment for a DL/I file has not been specified. |  | Environment creation
WRC0348 | The file FileName is empty. 5000 records are assumed. | An informational message is generated after an empty file has been loaded. The number of 5000 records is assumed. |  | File loading
WRC0356 | Method MethodName has been deactivated as field FieldName is no longer listed in file FileName but is still associated to a reduction rule. | The specified method has been deactivated. After a file structure change, a field that is no longer present in the new structure is associated to an association rule. | Check the reduction logic of the specified file and method, and apply the reduction criteria coherent with the new file structure. | Life Cycle
WRC0363 | Field FieldName of file FileName exceeds the maximum length for type FieldType. The next elaboration phase cannot manage it. | An internal elaboration, phase cannot manage the field. |  |
WRC0364 | Fixed point types are not supported; the field FieldName of file FileName will be treated as a 4-byte binary number. |  |  |
WRC0366 | The record length must be less than 32760. Record RecordNumber of file FileName will be discarded. | Data Express cannot handle files with record lengths greater than 32760. |  |
WRC0367 | JobName - Start elaboration | The specified job has started. |  |
WRC0368 | JobName - End elaboration | The specified job has ended. |  |
WRC0369 | An error occurred during the edit of string StringName in SourceName. | During the preparation of the job, job parameters cannot be substituted. |  |
WRC0370 | Error found in writing file FileName - File status: FileStatus. | Data Express is not able to write the specified file. | Verify that the file exists, there is space in the file, and that the user has authorizations on file. |
WRC0371 | Job JobName submitted. | Data Express has submitted the specified job. |  |
WRC0372 | JCL JCL-Name created. | Data Express has created the specified JCL. |  |
WRC0373 | The output file has more than 44 characters. File: FileName, Prefix: FilePrefix. | The output file is expected to be a MVS file, which has no more than 44 characters. |  |
WRC0374 | Error in reading file FileName - File status: FileStatus. | Data Express is not able to read the specified file. | Verify that the file exists and that the user has authorizations on file. |
WRC0375 | An abend has occurred. | Data Express has identified a batch phase that had an abnormal termination. | Check the MVS sysout to determine the reason for the abend. |
WRC0376 | More than 249 files have been found for method MethodName processing. | For each extraction job generated, no more than 249 files can be put in the same JCL. |  | Create Extraction Jobs
WRC0377 | Data set Data-Set-Name and data set Data-Set-Name have not been found in the SYSCOPYS table. Check the unload input file. | The DB2 system catalog information (SYSCOPYS table) does not contain the information about the specified image copybook data set. | Check the name of the specified image copybook data set. | Masking, subsetting
WRC0378 | The field FieldName has a length of 16 bytes, which assumes 31 digits. | It is assumed that a binary field of 16 bytes has 31 digits. |  | File loading
WRC0379 | An error occurred during the Level ID calculation. | An internal error occurred in the Level ID calculation. | Check previous messages during the calculation for more information. | File loading
WRC0380 | The process identifier ProcessID is not permitted for the job Load DB2 Table Information by Unload. | For the job Load DB2 Table Information by Unload, only access types R, U, and W are supported. | Verify the access type is appropriate for the process identifier you are using. | File loading
WRC0381 | The table name is not defined in the table-name statement in DCLGEN DCLGEN-Member. | In the parsing of the DCLGEN, the keyword DECLARE has not been found. |  | File loading
WRC0382 | The DCLGEN will be discarded. | Due to a previous error, the DCLGEN has not been processed. |  | File loading
WRC0383 | The creator has not been defined in the table-name statement in DCLGEN DCLGEN-Member. | In the parsing of the DCLGEN, the table creator has not been found. |  | File loading
WRC0384 | Unexpected column type ColumnType was found in DCLGEN-Member. | In the parsing of the DCLGEN, an unexpected or unhandled column type has been found. |  | File loading
WRC0385 | Unknown syntax was found in the table-name statement. Check the DCLGEN source DCLGEN-Member. | In the parsing of the DCLGEN, the table name has not been found. |  | File loading
WRC0386 | Error opening data set Data-Set-Name. | Data Express is not able to open the specified file. | Verify the file exists, there is space in the file, and that the user has authorizations on file. |
WRC0387 | Error closing file FileName - File status: FileStatus. | Data Express is not able to close the specified file. | Verify the file exists, there is space in the file, and that the user has authorizations on file. |
WRC0388 | DBD: DatabaseName | Information about the processed file is written in the job output. |  |
WRC0389 | File type: FileType | Information about the processed file is written in the job output. |  |
WRC0390 | File Name: FileName | Information about the processed file is written in the job output. |  |
WRC0391 | Version: FileVersion | Information about the processed file is written in the job output. |  |
WRC0392 | Format: FileFormat | Information about the processed file is written in the job output. |  |
WRC0393 | Records processed: x | Information about the processed file is written in the job output. |  |
WRC0394 | Skip factor: SkipFactor | Information about the processed file is written in the job output. |  |
WRC0395 | Unknown syntax was found in the input file at row StatementNumber. | Unexpected syntax has been found in the input file for the DB2 Catalog Synchronization phase. | Check the syntax listed in the Data Model Guide. | DB2 Catalog Synchronization
WRC0396 | Unable to retrieve the file image FileImage. | An internal error occurred during processing of the phase. | Check previous messages for more information. | File-copybook relationship analysis
WRC0397 | File File-Library-Name. FileName does not contain information for the field. The file was discarded. | Field level information is missing in the file interface. | Check the syntax of the interface in the Data Model Guide. | File loading
WRC0398 | The copybook library Copybook-Library-Name contains invalid characters. | The copybook name specified in the interface is incorrect. | Check the syntax of the interface in the Data Model Guide. | File loading
WRC0399 | A copybook name is required for copybook library Copybook-Library-Name. | The copybook name specified in the interface is incorrect. | Check the syntax of the interface in the Data Model Guide. | File loading
Parent topic: Error Messages


Mainframe Messages WRC0400 to WRC0499

Details Data Express mainframe messages in the specified message ID range.
Message ID | Message Text | Description | Resolution | Phase of Reference
WRC0400 | The copybook name Copybook-Library-Name contains invalid characters. | The copybook name specified in the interface is incorrect. | Check the syntax of the interface in the Data Model Guide. | File loading
WRC0401 | The I/O area I/O-Area-Name contains invalid characters. | The copybook I/O area name specified in the interface is incorrect. | Check the syntax of the interface in the Data Model Guide. | File loading
WRC0402 | The I/O area progressive I/O-Area-Progressive contains invalid characters. | The copybook I/O area name specified in the interface is incorrect. | Check the syntax of the interface in the Data Model Guide. | File loading
WRC0403 | Copybook library Copybook-Library-Name, copybook name CopybookName does not exist. | The copybook specified in the interface does not exist. | Check the existence of the copybook, and check the correctness of the copybook name and library. | File loading
WRC0404 | The copybook library Copybook-Library-Name, copybook name CopybookName has not been analyzed. | The copybook specified in the interface has not been analyzed by Data Express. | Check the analysis status of the copybook in Data Express. | File loading
WRC0405 | The copybook library Copybook-Library-Name, copybook name CopybookName, I/O area name I/O-Area-Name, I/O area progressive. I/O-Area-Progressive: the specified I/O area does not exist. | The copybook specified in the interface exists, but the specified I/O area is not consistent with the result of Data Express analysis. | Check the analysis status of the copybook in Data Express. | File loading
WRC0406 | File FileName has been already associated with a copybook. | The copybook specified in the interface cannot be assigned to the file because the file already has a copybook association. | Check the file status in Data Express, and delete the existing assignment if necessary. | File loading
WRC0407 | A record was found that references a blank copybook library. | The copybook library specified in the interface is incorrect. | Check the syntax of the interface in the Data Model Guide. | File loading
WRC0408 | Assumed maximum I/O area = 001 for PDS Copybook-Library-Name - copybook CopybookName. | The limit of I/O area numbers has been reached, so all fields are assigned to the progressive number 1. | Check the maximum number for the I/O area (parameter Maximum Progressive of I/O Area in "Analysis options"). | File loading
WRC0409 | The PDS Copybook-Library-Name contains x member(s). | An informational message is generated about the processed PDS. |  | Life cycle
WRC0410 | Starting analysis for include IncludeName. | An informational message is generated about the processed copybook. |  | File loading
WRC0411 | Starting analysis for copybook CopybookName. | An informational message is generated about the processed copybook. |  | File loading
WRC0412 | Start CPU Time = CPU-Time. | An informational message is generated about the processed file. |  | All batch phases
WRC0413 | End CPU Time = CPU-Time. | An informational message is generated about the processed file. |  | All batch phases
WRC0414 | No enabled field was found for the specified file. | The sampling has been submitted for the file, but no enabled fields exist for the file. | Check the sampling enabling flag in Data Express for the specified file. | Sampling
WRC0415 | More than 750 fields were found. | The internal limit of 750 fields for each file has been reached. | Only the first 750 fields are considered for masking and reduction. If you have more than 750 fields, it is possible you need to manually write a copybook containing the fields really to be processed. | All batch phases
WRC0416 | Table TableName is full. Extend it and run the job again. | The repository table specified in the message has reached its limit and can no longer accept record insertions. | Extend or reorganize the specified table. | Sampling
WRC0417 | The sampling could not be processed for field FieldName in file FileName because the field has more than 31 digits. | A product limit has been reached. Only the first 31 digits of the fields are sampled. |  | Sampling
WRC0418 | Version : 0 | An informational message is generated about the processed file. |  | All batch phases
WRC0419 | The file name was not found for file FileName. | The file name specified in the interface is incorrect. | Check the syntax of the interface in the Data Model Guide. | File loading
WRC0420 | Starting copybook/file association for file FileName. | An informational message is generated about the processed file. |  | File-Copybook association
WRC0421 | Unable to retrieve image for copybook CopybookName. | An internal error was found in a matching algorithm. |  | File-Copybook association
WRC0422 | No match was found. | An informational message is generated about the processed file. |  | File-Copybook association
WRC0423 | Unknown job JobName was found and discarded. | An unexpected job was found in the list of jobs to be submitted. |  | Job submission
WRC0424 | Unable to retrieve Machine ID/Company identifier for machine ID MachineID and company CompanyName. | An internal error occurred while retrieving machine ID and company information. | Check previous error messages for more information. | All batch phases
WRC0425 | No valid class is available. | An internal error occurred while retrieving machine ID and company information. | Check previous error messages for more information. | Field-Class assignment
WRC0426 | Field managed: FieldName. | An informational message is generated about the result of the processing. |  | Field-Class assignment
WRC0427 | Field assigned: FieldName. | An informational message is generated about the result of the processing. |  | Field-Class assignment
WRC0428 | No valid field was found for the selected options. | An informational message is generated about the result of the processing. |  | Field-Class assignment
WRC0429 | More than 249 files were found to process for selection. | The data masking process cannot handle more than 249 files. | Use parameters in order to create distinct processes for large a group of files in order to avoid this limit. | Data Masking
WRC0430 | An error was found in program ProgramName. | An internal error was found in a writing routine. | Check previous error messages for more information. | Data Masking, Data Subsetting
WRC0431 | Error in write: Data-Set-Name. Record length: x. | Data Express is not able to write to the specified file. | Verify the file exists, there is space in the data set and the user has authorizations on file. | Data Masking, Data Subsetting
WRC0432 | There are no valid indexes for table TableName. | An internal error occurred during file access. | Check previous error messages for more information. | Data Masking,
WRC0433 | There are no valid fields for the update table operation TableOperation. | An internal error occurred during file access. | Check previous error messages for more information. | Data Masking,
WRC0434 | Record number x has been converted. | An informational message is generated about the result of the processing. |  | Data Masking
WRC0435 | FieldName: the field type FieldType is not managed. | An internal error occurred during field type retrieval. | Check previous error messages for more information. | Data Masking, Data Subsetting
WRC0436 | 250 is the maximum number of elements for the file. | The data masking process of cannot handle more than 250 files | Use parameters in order to create distinct processes for a large group of files, in order to avoid this limit. | Data Masking, Data Subsetting
WRC0437 | An internal data set name was not found. | An internal error occurred during file access. | Check previous error messages for more information. | Data Masking
WRC0438 | Error in starting file FileName - File status FileStatus. | An internal error occurred during file access. | Check previous error messages for more information. | Data Masking
WRC0439 | Error in rewriting file FileName - File status FileStatus. | An internal error occurred during file access. | Check previous error messages for more information. | Data Masking
WRC0440 | Error in deleting file FileName - File status FileStatus. |  |  |
WRC0448 | Error closing data set Data-Set-Name. | Data Express is not able to close the specified file. | Verify the file status. | Data Masking, Data Subsetting
WRC0449 | Optimized access with index for range. Field: FieldName - Maximum value: x. | An informational message is generated about the file to be reduced. |  | Data Subsetting
WRC0450 | Optimized access with index for range. Field: FieldName - Maximum value: x. | An informational message is generated about the file to be reduced. |  | Data Subsetting
WRC0451 | Optimized access with index for range. Field: FieldName - Maximum value: x. | An informational message is generated about the file to be reduced. |  | Data Subsetting
WRC0452 | Optimized access + filter routine. Field: Field: FieldName - Value: x. | An informational message is generated about the file to be reduced. |  | Data Subsetting
WRC0453 | Error in parameter ParameterName. | An internal error occurred during log writing. | Check previous error messages for more information. | All phases
WRC0454 | Starting elaboration for file FileName. Output data set: Data-Set-Name. JCL: JCL-Name. | An informational message is generated about the processed file. |  | Data Subsetting
WRC0455 | The JCL JCL-Name does not exist. | The JCL specified in the message has been specified in the reduction condition, but could not be found by Data Express. | Verify the filter, and check if the JCL name is correctly written. | Data Subsetting
WRC0456 | Error in writing additional step. | An internal error occurred during JCL writing. | Check previous error messages for more information. | Data Subsetting
WRC0457 | Operation OperationName is not valid. | An internal error occurred while writing to the table ANENVLST. | Check previous error messages for more information. | Data Subsetting
WRC0458 | The JCL was not submitted. | An internal error occurred during JCL writing. | Check previous error messages for more information. | Data Subsetting
WRC0459 | Error open output: FileName. | Data Express is not able to open the specified file. | Verify the file exists, there is space in the file, and the user has authorizations on file. | Data Subsetting
WRC0460 | Starting elaboration for file: FileName. | An informational message is generated about the file to be reduced. |  | Data Subsetting
WRC0461 | DB2 Environment: EnvironmentName. | An informational message is generated about the file to be reduced. |  | Data Subsetting
WRC0462 | Input data set: Data-Set-Name. | An informational message is generated about the file to be reduced. |  | Data Subsetting
WRC0463 | Input data set type: Data-Set-Name. | An informational message is generated about the file to be reduced. |  | Data Subsetting
WRC0464 | Output data set: Data-Set-Name. | An informational message is generated about the file to be reduced. |  | Data Subsetting
WRC0465 | Successful elaboration for file: FileName. | An informational message is generated about the file to be reduced. |  | Data Subsetting
WRC0466 | The file FileName. does not exist. | An internal error occurred during the life cycle. | Check previous error messages for more information. | Life cycle
WRC0467 | Error reading: FileName. | Data Express is not able to read the specified file. | Verify the file exists and the user has authorizations on file. | Data Subsetting
WRC0468 | The called program ProgramName does not function properly. Copy it from source. | An internal error occurred data subsetting. | Check previous error messages for more information. | Data Subsetting
WRC0469 | Error writing: FileName. | Data Express is not able to write to the specified file. | Verify the file exists, there is space in the file, and the user has authorizations on file. | Data Subsetting
WRC0470 | Record length: x. | An informational message is generated about the file to be reduced. |  | Data Subsetting
WRC0471 | The DBD structure has been loaded with x element(s). | An informational message is generated about successful completion of the DBD structure analysis. |  | Data Subsetting
WRC0472 | The segment was not found in the DBD structure segment SegmentName. | The specified segment has not been found in the DBD structure. | Check the DBD source. | Data Subsetting
WRC0473 | data masking routine | An informational message is generated about access to the file to be reduced. |  | Data Subsetting
WRC0474 | Field name: FieldName | An informational message is generated about access to the file to be reduced. |  | Data Subsetting
WRC0475 | user exit program: ProgramName | An informational message is generated about access to the file to be reduced. |  | Data Subsetting
WRC0476 | x million records have been elaborated at Date Time. | An informational message is generated about the number of processed records needed to monitor the process status. |  | Data Subsetting
WRC0477 | Unable to retrieve fldrecid information. | An internal error occurred during data subsetting. The fldrecid information, which is the internal logical key that uniquely identifies a field within a file, is missing. | Check previous error messages for more information. | Data Subsetting
WRC0478 | An internal error occurred during a search of field details for field no. x. | An internal error occurred during data subsetting. Expected information was not found in the knowledge base. | Check previous error messages for more information. | Data Subsetting
WRC0479 | Error in retrieving FLTPRG on HSENVELB for bank. | An internal error occurred during data subsetting. | Check previous error messages for more information. | Data Subsetting
WRC0480 | Filter by value list. | An informational message is generated about access to the file to be reduced. |  | Data Subsetting
WRC0481 | Access is not optimized. The primary or alternate key is unavailable. | An informational message is generated about access to the file to be reduced. |  | Data Subsetting
WRC0482 | Optimized access | An informational message is generated about access to the file to be reduced. |  | Data Subsetting
WRC0483 | Access is not optimized. More than 37 values have been returned. | An informational message is generated about access to the file to be reduced. |  | Data Subsetting
WRC0484 | Filename DBD DL/I: FileName. | An informational message is generated about the name of the file containing the DLI/DBD to be analyzed. |  | Data Subsetting
WRC0485 | Error opening DL/I file FileName. | The specified data set name, containing the DL/I DBD cannot be opened. | Verify the file exists and the user has authorizations on file. | Data Subsetting
WRC0486 | The root segment for this DBD cannot be found. Provide at least one segment without any parent specification. | An internal error occurred while parsing the DL/I DBD. | Verify the syntax of the DL/I DBD. | Data Subsetting
WRC0487 | Cannot perform TableOperation on table TableName. SQLCODE SQL-Code. | An error occurred while accessing the specified DB2 table. | Check the specified DB2 table according to the reported SQLCODE. | All batch phases
WRC0488 | The data type DataType is unrecognized. | The specified data type is not supported by Data Express. |  | File loading
WRC0489 | No record found for table TableName. | A warning is generated indicating that the DB2 table you are loading is empty. |  | File loading
WRC0490 | SQLCODE on operation : SQL-Code. | An error occurred while accessing the specified DB2 table. | Check the specified DB2 table according to the reported SQLCODE. | All batch phases
WRC0491 | The fields of table TableName cannot be accessed. SQLCODE SQL-Code. | An error occurred while accessing the specified DB2 table. | Check the specified DB2 table according to the reported SQLCODE. | File loading
WRC0492 | More than 500 classes were found. | The limit for the number of handled classes has been reached. |  | Class-Field assignment
WRC0493 | More than 40 naming conventions were found for class ClassName. | The limit for the number of handled class naming conventions has been reached. |  | Class-Field assignment
WRC0494 | File FileName has a different file type. | There is a difference between the interface used by this phase and the information stored in the knowledge base. | Update the interface according to the content of the knowledge base. | Import classification from data dictionary
WRC0495 | File FileName has a different file library. | There is a difference between the interface used by this phase and the information stored in the knowledge base. | Update the interface according to the content of the knowledge base. | Import classification from data dictionary
WRC0496 | File FileName has a different file name. | There is a difference between the interface used by this phase and the information stored in the knowledge base. | Update the interface according to the content of the knowledge base. | Import classification from data dictionary
WRC0497 | File FileName has a different file version. | There is a difference between the interface used by this phase and the information stored in the knowledge base. | Update the interface according to the content of the knowledge base. | Import classification from data dictionary
WRC0498 | File FileName has a different record format. | There is a difference between the interface used by this phase and the information stored in the knowledge base. | Update the interface according to the content of the knowledge base. | Import classification from data dictionary
WRC0499 | File FileName has a different application ID. | There is a difference between the interface used by this phase and the information stored in the knowledge base. | Update the interface according to the content of the knowledge base. | Import classification from data dictionary
Parent topic: Error Messages


Mainframe Messages WRC0500 to WRC0599

Details Data Express mainframe messages in the specified message ID range.
Message ID | Message Text | Description | Resolution | Phase of Reference
WRC0500 | File FileName found more than one time. | An internal product error occurred during file retrieval. | Check previous error messages for more information. | Import classification from data dictionary, Import classification from referential integrity, Import method from referential integrity
WRC0501 | File FileName not found. | An internal product error occurred during file retrieval. | Check previous error messages for more information. | Import classification from data dictionary, Import classification from referential integrity, Import method from referential integrity
WRC0502 | The field FieldName of file FileName does not exist. | There is a difference between the interface used by this phase and the information stored in the knowledge base. | Update the interface according to the content of the knowledge base. | Import classification from data dictionary
WRC0503 | The field FieldName of file FileName already has an estimated class. | The assignment is ignored because there is already an estimated class. | Clear the estimated class if you want to associate the one coming from the data dictionary. | Import classification from data dictionary
WRC0504 | Class ClassName is associated to field FieldName of file FileName. | The assignment is ignored because there is already an associated class. | Clear the associated class if you want to associate the one coming from the data dictionary. | Import classification from data dictionary
WRC0505 | Class ClassName has not been loaded. |  |  |
WRC0506 | File FileName - class ClassName: a class description is required. | Required data is missing. | Enter the required value accordingly. | Import classification from data dictionary
WRC0507 | File FileName - class ClassName: a class type is required. | Required data is missing. | Enter the required value accordingly. | Import classification from data dictionary
WRC0508 | File FileName - class ClassName: the maximum length must be greater than or equal to the minimum length. | There is a difference between the interface used by this phase and the information stored in the knowledge base. | Update the interface according to the content of the knowledge base. | Import classification from data dictionary
WRC0509 | File FileName - class ClassName: the number of digits for the minimum length value must be less than or equal to 31. | There is a difference between the interface used by this phase and the information stored in the knowledge base. | Update the interface according to the content of the knowledge base. | Import classification from data dictionary
WRC0510 | File FileName - class ClassName: the number of digits for the maximum length value must be less than or equal to 31. | There is a difference between the interface used by this phase and the information stored in the knowledge base. | Update the interface according to the content of the knowledge base. | Import classification from data dictionary
WRC0511 | File FileName - class ClassName: the number of digits for the minimum integer value must be less than or equal to 31. | There is a difference between the interface used by this phase and the information stored in the knowledge base. | Update the interface according to the content of the knowledge base. | Import classification from data dictionary
WRC0512 | File FileName - class ClassName: the number of digits for the maximum integer value must be less than or equal to 31. | There is a difference between the interface used by this phase and the information stored in the knowledge base. | Update the interface according to the content of the knowledge base. | Import classification from data dictionary
WRC0513 | File FileName - class ClassName: the number of digits for the minimum integer value must be less than or equal to 31. | There is a difference between the interface used by this phase and the information stored in the knowledge base. | Update the interface according to the content of the knowledge base. | Import classification from data dictionary
WRC0514 | File FileName - class ClassName: the number of digits for the minimum decimal value must be less than or equal to 31. | There is a difference between the interface used by this phase and the information stored in the knowledge base. | Update the interface according to the content of the knowledge base. | Import classification from data dictionary
WRC0515 | File FileName - classClassName: the number of digits for the maximum decimal value must be less than or equal to 31. | There is a difference between the interface used by this phase and the information stored in the knowledge base. | Update the interface according to the content of the knowledge base. | Import classification from data dictionary
WRC0516 | File FileName - class ClassName: the minimum decimal number must be less than or equal to the maximum decimal number. | There is a difference between the interface used by this phase and the information stored in the knowledge base. | Update the interface according to the content of the knowledge base. | Import classification from data dictionary
WRC0517 | File FileName - class ClassName: the number of digits for both the minimum integer and minimum decimal values must be less than or equal to 31. | There is a difference between the interface used by this phase and the information stored in the knowledge base. | Update the interface according to the content of the knowledge base. | Import classification from data dictionary
WRC0518 | File FileName - class ClassName: a mismatch was found between the number of digits for the minimum length, integer, and decimal values. | There is a difference between the interface used by this phase and the information stored in the knowledge base. | Update the interface according to the content of the knowledge base. | Import classification from data dictionary
WRC0519 | File FileName - class ClassName: a mismatch was found between the minimum length and the minimum integer. | There is a difference between the interface used by this phase and the information stored in the knowledge base. | Update the interface according to the content of the knowledge base. | Import classification from data dictionary
WRC0520 | File FileName - class ClassName: the number of digits for both the maximum integer and maximum decimal values must be less or equal to 31. | There is a difference between the interface used by this phase and the information stored in the knowledge base. | Update the interface according to the content of the knowledge base. | Import classification from data dictionary
WRC0521 | File FileName - class ClassName: a mismatch was found between the number of digits for the maximum length and for both the maximum integer and maximum decimal values. | There is a difference between the interface used by this phase and the information stored in the knowledge base. | Update the interface according to the content of the knowledge base. | Import classification from data dictionary
WRC0522 | File FileName - class ClassName: the number of digits in the maximum length is less than the maximum integer. | There is a difference between the interface used by this phase and the information stored in the knowledge base. | Update the interface according to the content of the knowledge base. | Import classification from data dictionary
WRC0524 | The job has been removed from the table HSURDJOB. | A warning is generated indicating that Data Express is not able to update information on the HSURDJOB table. |  | Job submission
WRC0525 | The file/copybook association has been lost for copybook CopybookName - I/O area name: I/O-Area-Name - I/O area progressive: I/O-Area-Progressive. | The structure of the copybook has changed, and the file assignment must be verified. | Check if the file associated to the copybook needs to be reloaded in knowledge base. | Life cycle
WRC0526 | The copybook has been found, but its I/O area does not correspond with the old analysis. Format: File-Record-Format - I/O area progressive: I/O-Area-Progressive. | The structure of the copybook has changed, and the file assignment must be verified. | Check if the file associated to the copybook needs to be reloaded in knowledge base. | Life cycle
WRC0527 | New I/O area found for copybook CopybookName - I/O area name: I/O-Area-Name - I/O area progressive: I/O-Area-Progressive. | The structure of the copybook has changed, and the file assignment must be verified. | Check if the file associated to the copybook needs to be reloaded in knowledge base. | Life cycle
WRC0528 | Unknown syntax was found in the table-name statement. Check the source DCLGEN-Member. | The table name could not be found during the analysis of the DCLGEN. |  | File loading
WRC0529 | Loading file FileName. | An informational message is generated indicating that the life cycle is loading information for a file in the knowledge base. |  | Life cycle
WRC0530 | A combined field was found. | An informational message is generated indicating that the life cycle is using a backup for a combined field. |  | Life cycle
WRC0531 | Backup mcrecid: mcrecid - File ID: FileID - Field ID: FieldID. | An informational message is generated indicating that the life cycle is using a backup for a combined field. |  | Life cycle
WRC0532 | The unload file FileName was not found. | Data Express is not able to read the specified file. | Verify the file exists and the user has authorizations on file. | Life cycle
WRC0533 | More than 32 fields in a relationship were found. | Data Express is not able to handle referential integrity relationships involving more than 32 fields. |  | Import classification from referential integrity, Import method from referential integrity
WRC0534 | Column Column-Name - assign class ClassName was not found in table TableName. Check the Referential Integrity interface. | A mismatch was found in classes for the specified relationship. | Check the interface. | Import classification from referential integrity, Import method from referential integrity
WRC0535 | Column Column-Name - estimated class ClassName was not found in table TableName. Check the Referential Integrity interface. | A mismatch was found in classes for the specified relationship. | Check the interface. | Import classification from referential integrity, Import method from referential integrity
WRC0536 | Column Column-Name - class ClassName was not found in table TableName. Check the Referential Integrity interface. | A mismatch was found in classes for the specified relationship. | Check the interface. | Import classification from referential integrity, Import method from referential integrity
WRC0537 | More than 9999 classes were found. | Data Express is not able to handle more than 9999 classes. |  | Import classification from referential integrity
WRC0538 | Field FieldName of file FileName was found more than one time. | An internal product error occurred during field information retrieval. | Check previous error messages for more information. | Import classification from referential integrity, Import method from referential integrity
WRC0539 | Field no. x of file FileName was found more than one time. | An internal product error occurred during field information retrieval. | Check previous error messages for more information. | Import classification from referential integrity
WRC0540 | An error occurred during retrieval of the maximum FLDRECID from the table ANURDFLF. | An internal product error occurred during field information retrieval. | Check previous error messages for more information. | Import classification from referential integrity
WRC0541 | A mismatch between classes was found. | An internal product error occurred during field information retrieval. | Check previous error messages for more information. | Import classification from referential integrity, Import method from referential integrity
WRC0542 | Field FieldName of file FileName contains class ClassName. | An internal product error occurred during field information retrieval. | Check previous error messages for more information. | Import classification from referential integrity
WRC0543 | A difference in type or length was found between field FieldName of file FileName and field FieldName of file FileName. The relationship will be discarded. | The current relationship is not handled by Data Express. The interface of referential integrity establishes relationships between fields not having the same characteristics. | Check the interface of referential integrity to see if the relationships are correct. | Import classification from referential integrity
WRC0544 | More than 15 columns were found during a check for matching indexes. 15 columns will be assumed. | Data Express is not able to check optimization for relationships involving more than 15 columns. For those relationships, the control is not executed. | The optimization check does not impact the creation of the method, so the message can be ignored. But if those relationships follow an index, it is recommended that optimization be checked in order to ensure data subsetting performance. | Import method from referential integrity
WRC0545 | An index was not found for parent TableName. | The specified relationship cannot be optimized by Data Subset Extraction, because it is not coherent with indexes. | Check the interface of referential integrity to see if the relationships are correct. | Import method from referential integrity
WRC0546 | An index was not found for child TableName. | The specified relationship cannot be optimized by Data Subset Extraction, because it is not coherent with indexes. | Check the interface of referential integrity to see if the relationships are correct. | Import method from referential integrity
WRC0547 | A mismatch was found. | An internal product error occurred during field information retrieval. | Check previous error messages for more information. | Import method from referential integrity
WRC0548 | Field FieldName of file FileName contains class ClassName. | An internal product error occurred during field information retrieval. | Check previous error messages for more information. | Import method from referential integrity
WRC0549 | Recursive relationship for: Parent-Table-Name - Child-Table-Name. | A recursive referential integrity relationship has been found. | Check the interface of referential integrity to see if the relationships are correct. If they are correct, check the generated method and change it with the client interface if needed. | Import classification from referential integrity, Import method from referential integrity
WRC0550 | File FileName has more than 10 filters. | Data Express supports up to 10 filters (in OR conditions) for each file. More than 10 different relationships and consequently more than 10 filters have been found. | Check the interface of referential integrity to see if the relationships are correct. If they are correct, check the generated method and change it with the client interface if needed. | Import classification from referential integrity, Import method from referential integrity
WRC0551 | Maximum number of elem for token table has been reached, check File Name source. | The maximum number of Data Store elements has been reached. | Check the File. | Create CSV file from Referential Integrity
WRC0552 | Program Name - Find error in parameter. RC: Return Code. | An error during the reading of a line in the source file was found which contains the description of Referential Integrity. | Check the File. | Create CSV file from Referential Integrity
WRC0553 | The maximum number of source lines for referential integrity definition has been reached. | The maximum number of source lines for referential integrity definition has been reached. | Check the file and if it is possible, delete some rows. | Create CSV file from Referential Integrity
WRC0554 | Unable to create SQL statement. | There is a problem related to the creation of the DELETE SQL statement. | Check the Method used for this process. | Deletion Job Generation (from toolkit)
WRC0555 | Del/Def PDS Name created for file File Name. | The PDS which contains the JCL created for the LISTCAT has been created. | This is not an error message. | Creation Delete Define Sources (from toolkit).
WRC0556 | Primary quantity not found. | The primary quantity cannot be found. | Check the parameters inserted into the Job mask. | Creation Delete Define Sources
WRC0557 | Secondary quantity not found. | The secondary quantity cannot be found. | Check the parameters inserted into the Job mask. | Creation Delete Define Sources
WRC0558 | The current string String Name will not be replaced. | Cannot replace all characters included in the stringfound in the secondary quantity. | Check the parameters inserted into the Job mask. | Creation Delete Define Sources
WRC0559 | Error reading data set Data Set name. | Cannot read the ADABAS FDT file. | Check if the file exists or if it is already open. | Analysis of ADABAS FDT Report
WRC0560 | Database name not found. | Cannot read the ADABAS FDT Data Base name from the FDT report. | Check if the FDT report has been created by the ADAREP. | Analysis of ADABAS FDT Report
WRC0561 | Database number not found. | Cannot read the ADABAS FDT Data Base number from the FDT report. | Check if the FDT report has been created by the ADAREP. | Analysis of ADABAS FDT Report
WRC0562 | File name not found. | Cannot read the File name from the ADABAS FDT report. | Check if the FDT report has been created by the ADAREP. | Analysis of ADABAS FDT Report
WRC0563 | Table with more than 255 partitions. | There are more than 255 partitions for the DB2 table. | Check if the partitions of the DB2 table should be modified. | General Creation of a DB2 DDL
WRC0564 | Found more than 100 indexes. | There are more than 100 indexes for the DB2 table. | Check if the index of the DB2 table should be modified. | General Creation of a DB2 DDL
WRC0565 | Found more than 20 keys. | There are more than 20 keys for the DB2 table. | Check if the keys of the DB2 table should be modified. | General Creation of a DB2 DDL
WRC0566 | Found more than 100 partitions. | There are more than 100 partitions for the DB2 table. | Check if the partitions of the DB2 table should be reduced. | General Creation of a DB2 DDL.
WRC0567 | The following string exceeds the limit of 1024 bytes. It will be truncated - String value: String Value. | There are more than 1024 bytes into the analyzed string. | Check if the string should be reduced. | General Creation of a DB2 DDL
WRC0568 | Single column Column Name is not supported, updateauth forced to all columns. | The update and referential permission can only be assigned to a single column. |  | General Creation of a DB2 DDL
WRC0569 | Job Job Name created. | A Job has been created. | This is not an error message. | At the end of job submission
WRC0570 | The input VSAM file name File Name will not be replaced. | The Input VSAM file name is wrong. | Check the VSAM file name. | Load VSAM file
WRC0571 | Record size not found or equal zero. | The record size is missing or is equal to 0. | Check the size of the record of the VSAM file. | Load VSAM file
Parent topic: Error Messages


Mainframe Messages WRC2000 to WRC4999

Details Data Express mainframe messages in the specified message ID range.
Message ID | Message Text | Description | Resolution | Phase of Reference
WRC2022 | The Data Express license has expired. | The product license has expired. | Contact Micro Focus Product Support for password verification or license extension. | All batch phases
WRC2037 | The phase is not enabled. | An internal error occurred during an enabling procedure. | Make sure that you have entered the correct password. If the problem persists, contact Micro Focus Product Support for verification of your product license. | All batch phases
WRC2039 | The Sampling phase is not enabled. | The product has not been enabled to perform the Sampling phase. | Make sure that you have entered the correct password. If the problem persists, contact Micro Focus Product Support for verification of your product license. | Sampling
WRC2040 | The Copybook/File Association phase is not enabled. | The product has not been enabled to perform the Copybook/File Association phase. | Make sure that you have entered the correct password. If the problem persists, contact Micro Focus Product Support for verification of your product license. | Copybook/file association
WRC2041 | The Data Masking phase is not enabled. | The product has not been enabled to perform the Data Masking phase. | Make sure that you have entered the correct password. If the problem persists, contact Micro Focus Product Support for verification of your product license. | Data masking
WRC2042 | The Data Subsetting phase is not enabled. | The product has not been enabled to perform the Data Subsetting phase. | Make sure that you have entered the correct password. If the problem persists, contact Micro Focus Product Support for verification of your product license. | Data subsetting
WRC2044 | The Life Cycle phase is not enabled. | The product has not been enabled to perform the Life Cycle phase. | Make sure that you have entered the correct password. If the problem persists, contact Micro Focus Product Support for verification of your product license. | Life cycle
WRC3024 | The From value must precede the To value. | The message displayed is a result of a check on the entered data. |  |
WRC3029 | The field contains an incorrect user exit value of x. |  |  |
WRC3033 | x changed file(s) have been found. | An informational message showing the number of changed files. |  | Life cycle
WRC3034 | x copybook(s) that have been changed and are not associated have been found. | An informational message showing the number of changed copybooks not associated to a file. |  | Life cycle
WRC3035 | x new copybook(s) have been found. | An informational message showing the number of new copybooks found. |  | Life cycle
WRC3036 | x new file(s) have been found. | An informational message showing the number of new files found. |  | Life cycle
WRC3037 | x file(s) with changed fields that do not require confirmation have been found. | An informational message showing the number of changed fields that do not require manual handling. |  | Life cycle
WRC3038 | x file(s) with changed fields that require confirmation have been found. | An informational message showing the number of changed fields that require manual handling using Work with changed fields. |  | Life cycle
WRC3039 | x file(s) have been updated. | An informational message showing the number of fully processed files. |  | Life cycle
WRC3040 | x copybook(s) have been updated. | An informational message showing the number of fully processed copybooks. |  | Life cycle
WRC4020 | The segment Segment Name was found in the DBD structure DBD Name but it was not found in the Knowledge Base. | The DL/I segment has been deleted from the Data Express knowledge base or the DL/I unload File has been changed between cataloging and submitting the Job. | Verify that the DBD structure is cataloged and if the segment has been deleted or the upload file has been changed it is necessary to reload all the DBD structure. | Test Environment Creation
WRC4021 | The segment Segment Name belonging to the DBD structure DBD Name was not found in the Method Method Name. | The DL/I segment has been excluded from the Method creation. | Verify the Method steps and if the segment has been excluded it is necessary to create a new method with the excluded segment. | Test Environment Creation
WRC4022 | The field segment name Segment Name contains invalid characters. | The field segment name of input file for the "load from external interface" contains invalid characters. | Verify the characters used for the field segment name. | Load From External Interface
WRC4023 | The user does not have privileges for table Table Name. | The user submitting the JCL does not have proper privileges for the DB2 table shown in the message. | Modify privileges to give Public Grant to the Table. | Load Sequential File and Load Impsrc For Life Cycle.
WRC4024 | The length of combined data elements combined name in table Table Name cannot be longer than 230. | The maximum length of the combined fields is 230 digits. | Create an exit routine in order to manage the longer combined data elements. | Data Subsetting Sampling
WRC4025 | Unsupported Data Type for table Table Name. | There are unsupported DB2 data types in managed table. | Data Express does not support some DB2 Data Types for Masking, Subsetting, Sampling via direct Access. It is necessary to Unload the table data into a SEQUENTIAL file. | Data Subsetting, Sampling, and Masking
WRC4026 | Sampling is not supported for ADABAS. | The sampling process is not supported for ADABAS data base. |  | Data Sampling
WRC4027 | File Datastore Name - class Class Name: a class short description is required. | The class short description is missing. | Insert the class short description into the Field DRCLATYPE of "Import Classification from Data Dictionary" sequential file. | Import Classification from Data Dictionary
WRC4028 | Unexpected syntax was found in SYSPUNCH SYSPUNCH Name during parsing of row with this statement SYSPUNCH Statement. | The BMC SYSPUNCH parser has encountered unsupported syntax in a SYSPUNCH statement. | Correct the SYSPUNCH statement. | Data Inventory
WRC4029 | No file found containing class Class Name. Check JCL input class. | The referential integrity file used by the JCL Import Method contains an invalid input class. | Correct the referential integrity file. | Data Subsetting
WRC4030 | ParentANDChildName field ParentANDChildField, file ParentDataStoreANDChildDataStore" | The referential integrity file contains a relationship in which a Parent data store its Child data store are recursive. | In the referential integrity file, identify the affected parent and child tables and correct the relationship. | Data Subsetting
WRC4031 | Method Method Name has not been successfully updated, please check to see if the method exists. | The method has been deleted or it has not been created. | Check the method name inserted into Job or uncheck the new button "Update method before submit the JCL". | Data Subsetting
Parent topic: Error Messages


Data Express Extension Messages

Message Number | Description | Resolution | Type
DE-0050 | The DML statement StatementName specified has been ignored. |  | Internal error
DE-0100 | Initialization routines called twice. | Contact Micro Focus Product Support for technical assistance. | Internal error
DE-0101 | Unable to allocate from Connection Pool. | Contact Micro Focus Product Support for technical assistance. | Internal error
DE-0102 | Unable to allocate from Statement Pool. | Contact Micro Focus Product Support for technical assistance. | Internal error
DE-0104 | Internal error (dxestart). MethodCode, MethodDescription, mcrecid, testrun, Dbname schema entry missing from methodfile. | Contact Micro Focus Product Support for technical assistance. | Internal error
DE-0105 | Internal error reading indicator Indicator, value Value, read value Value. | Contact Micro Focus Product Support for technical assistance. | Internal error
DE-0106 | Internal error reading length. | Contact Micro Focus Product Support for technical assistance. | Internal error
DE-0107 | Internal error reading data field FieldName: different data size, old Size new Size. | Contact Micro Focus Product Support for technical assistance. | Internal error
DE-0108 | Internal error reading data. | Contact Micro Focus Product Support for technical assistance. | Internal error
DE-0109 | Internal error reading ROWID. | Contact Micro Focus Product Support for technical assistance. | Internal error
DE-0110 | Internal error writing indicator. | Contact Micro Focus Product Support for technical assistance. | Internal error
DE-0111 | Internal error writing length. | Contact Micro Focus Product Support for technical assistance. | Internal error
DE-0112 | Internal error writing data field FieldName: larger data size Size, max Size. | Contact Micro Focus Product Support for technical assistance. | Internal error
DE-0113 | Internal error writing data. | Contact Micro Focus Product Support for technical assistance. | Internal error
DE-0114 | Internal error writing ROWID. | Contact Micro Focus Product Support for technical assistance. | Internal error
DE-0115 | Internal error (dxefinal). Unable to find the TableName in the chain. | Contact Micro Focus Product Support for technical assistance. | Internal error
DE-0126 | Internal error (dxefinal). Unable to find TotalRec in the chain. | Contact Micro Focus Product Support for technical assistance. | Internal error
DE-0127 | Internal error (dxefinal). Unable to find the ProcessedRec in the chain. | Contact Micro Focus Product Support for technical assistance. | Internal error
DE-0128 | Internal error (dxemasking). Error reading table definition from stdin. | Contact Micro Focus Product Support for technical assistance. | Internal error
DE-0129 | Internal error (dxeextracter). Incorrect column count: read Count, received Count. | Contact Micro Focus Product Support for technical assistance. | Internal error
DE-0130 | Internal error writing to progress monitoring file. | Contact Micro Focus Product Support for technical assistance. | Internal error
DE-0131 | Internal error reading progress monitoring file, table TableName. | Contact Micro Focus Product Support for technical assistance. | Internal error
DE-0132 | Internal error. Unknown column type ( Type ) on CREATE TABLE. | Contact Micro Focus Product Support for technical assistance. | Internal error
DE-0133 | Internal error allocating memory. | Contact Micro Focus Product Support for technical assistance. | Internal error
DE-0200 | Error reading configuration file ( FileName ). |  | Configuration files
DE-0201 | Unable to open file FileName. | For more information, see the Extension Technology Directories section in the Getting Started with Distributed Data Stores Guide. | Configuration files
DE-0202 | Error reading database connection criteria from MethodConnectionFile for data source DataSourceName. | Contact Micro Focus Product Support, providing the method.rc file. | Configuration files
DE-0203 | Invalid configuration file: FileName. | Ensure that the RDBMS, DBNAME, DBOWNER, USER, and PSWD settings are correct. For more information, see the section Masking Routines that Access Replacement Lookup Data of your Getting Started with Distributed Data Stores guide. | Configuration files
DE-0204 | Unable to retrieve connection criteria for the look-up data from hdblayer.rc. | View the log file for further details. | Configuration files
DE-0205 | Invalid configuration specification, Specification. |  | Configuration files
DE-0206 | Internal error (dxeextracter). Name data element name not found in the table/filter definition. | Contact Micro Focus Product Support for technical assistance. | Configuration files
DE-0207 | Internal error. Cannot determine the definition of the combined data element DataElement. | Contact Micro Focus Product Support for technical assistance. | Configuration files
DE-0208 | Internal error. Inconsistent data within elab.txt - number of fields exceeded ( Fields ). | Contact Micro Focus Product Support for technical assistance. | Configuration files
DE-0209 | Internal error. Item has incorrect format. | Contact Micro Focus Product Support for technical assistance. | Configuration files
DE-0210 | Internal error. Sampling input file, FileName, is not in the correct format. | Contact Micro Focus Product Support for technical assistance. | Configuration files
DE-0211 | Internal error. Invalid filter type: ( Type ). | Contact Micro Focus Product Support for technical assistance. | Configuration files
DE-0212 | Invalid record found in SQL Statement File, SQL Statement. Expected format isschema.owner~sql_statementwhere the SQL Statement must use a qualified table name. |  | Configuration files
DE-0213 | Error reading Data Express configuration file(%s). 'Text Output:' parameter is missing. \n |  | Configuration files
DE-0214 | Error reading Data Express configuration file(%s). 'Data Separator:' parameter is missing.\n |  | Configuration files
DE-0215 | Error reading Data Express configuration file(%s). 'Fields Enclosed By:' parameter is missing.\n |  | Configuration files
DE-0216 | Error reading Data Express configuration file(%s). 'LogLevel:' parameter is missing.\n |  | Configuration files
DE-0217 | Error reading Data Express configuration file(%s). 'CSV Data Store:' parameter is missing.\n |  | Configuration files
DE-0218 | Error reading Data Express configuration file(%s). 'COBOL Masking:' parameter is missing.\n |  | Configuration files
DE-0219 | Error reading Data Express configuration file(%s). 'Validate Target Table:' parameter is missing.\n |  | Configuration files
DE-0220 | Error reading Data Express configuration file(%s). 'Validate Source Schema:' parameter is missing.\n |  | Configuration files
DE-0221 | Error reading Data Express configuration file(%s). 'Enable Logging:' parameter is missing.\n |  | Configuration files
DE-0222 | Error reading Data Express configuration file(%s). 'Always Drop Table:' parameter is missing.\n |  | Configuration files
DE-0223 | Error reading Data Express configuration file(%s). 'Hide Progress:' parameter is missing.\n |  | Configuration files
DE-0224 | Error reading Data Express configuration file(%s). 'Hide Log Message:' parameter is missing.\n |  | Configuration files
DE-0225 | Error reading Data Express configuration file(%s). 'Use Extended C Masking:' parameter is missing.\n |  | Configuration files
DE-0226 | Error reading Data Express configuration file(%s). 'Identify Externally:' parameter is missing.\n |  | Configuration files
DE-0230 | Error initializing the Oracle environment. | Ensure that any environment variables required for working with Oracle are set. | Initialization
DE-0231 | Unable to load the ODBC Driver Manager. | Ensure that your ODBC environment is configured correctly. Refer to your ODBC Driver Manager documentation for details. | Initialization
DE-0232 | Unable to load Oracle client-side libraries. | Ensure that your Oracle environment is configured correctly. Refer to your Oracle documentation for details. | Initialization
DE-0233 | Unable to initialize the ODBC/database environment. |  | Initialization
DE-0234 | The COBOL environment needs to be available in order to process the masking routines. The method has not been executed. | Ensure that your COBOL environment is configured correctly. Refer to Net Express (Windows) or Server Express (UNIX) documentation for details. | Initialization
DE-0235 | Unable to load the masking library: Library. The method has not been executed. |  | Initialization
DE-0236 | Unable to load masking routine Routine. The method has not been executed. |  | Initialization
DE-0250 | Internal error. Data Express does not recognize the data type DataType for column ColumnName. | Contact Micro Focus Product Support for technical assistance. | ODBC Data Type
DE-0251 | Data type for column ColumnName ( DataTypeName ) not supported with COBOL masking. |  | ODBC Data Type
DE-0252 | Column ColumnName (ODBC type type ) in table TableName not supported for sampling. |  | ODBC Data Type
DE-0250 | Internal error. Data Express does not recognise data type DataType for column ColumnName. | Contact Micro Focus Product Support for technical assistance. | ORACLE Data Type
DE-0251 | Data type for column ColumnName ( DataTypeName ) not supported with COBOL masking. |  | ORACLE Data Type
DE-0252 | Column ColumnName (Oracle type type ) in table TableName not supported for sampling. |  | ORACLE Data Type
DE-0253 | Unable to process the table TableName due to unsupported data types. |  | ORACLE Data Type
DE-0300 | Use of the same source and target schema is not supported. | Execute the Distributed Exporter again, but select a different target schema. | Logic
DE-0301 | The table TableName already exists within the data store identified by DataStoreName. | To resolve this issue, change the configuration option AlwaysDropTable, defined in dxeconfig.cfg, to Y. | Logic
DE-0302 | The run-time Knowledge Base table TableName. SchemaName does not exist within the data store identified by DataStoreName. | Your DBA must create the run-time Knowledge Base tables and indexes prior to invoking Data Express . Refer your DBA to http://community.microfocus.com/microfocus/mainframe_solutions/enterprise_developer_21/w/knowledge_base/23153.dba-tasks-for-working-with-the-data-express-extension-technology.aspx for details. | Logic
DE-0303 | The method MethodName has no Output Class, and you have selected not to Write Output, so there is nothing to process. | Update the method definition within Data Subset Extraction to either specify an output class or to ensure that Write Output is selected. | Logic
DE-0304 | Error populating the Data Express Knowledge Base. | Review and correct any SQL errors reported within the log file, within the log subdirectory, in addition to the schema permissions within your database. | Logic
DE-0305 | Unable to SELECT from table in the source database. | Ensure that the database permissions provide the '%s' user with SELECT privileges. | Logic
DE-0306 | Unable to access table TableName in the target database. | Ensure that the database permissions provide the '%s' user with CREATE TABLE, SELECT and INSERT privileges. | Logic
DE-0307 | No data elements to update for table TableName. |  | Logic
DE-0308 | Unable to read original data row DataRow. |  | Logic
DE-0309 | Unable to read data, row Row. |  | Logic
DE-0310 | Data Express is unable to process Columns columns. The limit is Columns columns. Additional columns will be ignored. |  | Logic
DE-0311 | The Data Express Oracle Extension does not support processing data from multiple data stores. | Reduce the scope of your current method to only include tables from a single data store. | Logic
DE-0312 | Invalid directory: Directory. |  | Logic
DE-0313 | Data Express is already processing from RootDataDirectory. |  | Logic
DE-0314 | Data Express processing has been interrupted. |  | Logic
DE-0315 | No data fetched from table TableName. |  | Logic
DE-0316 | Usage: dxestart [-V] [ -d directory ]. |  | Logic
DE-0317 | Error row Row executing statement Statement. |  | Logic
DE-0318 | The target table definition does not match the source table definition. The method will not be executed. | Change the configuration option Always Drop Table to Y within config/dxeconfig.cfg, and re-execute dxestart. | Logic
DE-0319 | COBOL masking routine Routine terminated prematurely. |  | Logic
DE-0320 | Error returned from Data Express Module module. |  | Logic
DE-0321 | Error processing BFILE data. No path specified. |  | Logic
DE-0322 | Failed to connect to the data store identified by DataStoreName. |  | Logic
DE-0323 | Errors found during Data Express processing. See ErrorLogFile for details. |  | Logic
DE-0324 | Internal error. Incorrect data definition for column name ColumnName, received ColumnNameLength. | Contact Micro Focus Product Support for technical assistance. | Logic
Parent topic: Error Messages


Front End Guide

The guide provides step-by-step instructions to help you analyze and handle modifications of the data within the data sets and data stores of the applications running in your environment. With Data Express, you can verify the integrity of data within databases of applications that are either developed internally or purchased by external suppliers.

The scope of Data Express is to allow users to consult and maintain the Knowledge Base obtained to be able to analyze their own databases with a simple user interface which highlights:

  • Copybooks associated with the data store (Data Express for z/OS only.)
  • The corresponding record layout
  • The distribution of values in the data elements

Data Express contains the following elements:

  • Setup processing in the mainframe environment that automatically identifies data-store traces.
  • An engine in the mainframe environment that submits jobs for file analysis and sampling functions. This respectively allows you to verify and evaluate the integrity of data (or the jobs of other functions), such as the data masking and test environment creation functions. (Data Express for z/OS only.)
  • The evaluation of results using Data Express on a PC.

This guide provides the product configuration procedures and indicates the steps required to analyze the data stores that make up your application.

  • General Features
  • Considerations
  • Getting Started with Data Builder
  • Main Window - Work with Data Stores
  • Work with Data Store Window
  • Assign an Application ID Window
  • Add to Combined Data Element Window
  • Work with Classes
  • Work with Machine IDs
  • Work with Companies
  • Work with Application IDs
  • Work with Routines
  • Work with Process IDs
  • Work with Data Elements
  • Work with Jobs
  • New Custom Query Wizard
  • Options
  • File Properties
  • Filter by Data Store
  • Compare Layouts - Copybook
  • Sample Graph - Data Sampling
  • Show Synthetic Data Element Contents
  • Work with Copybooks
  • Distributed Loader
  • Appendix A. Standard Mode
  • Appendix B. Client / Server Mode
Parent topic: Data Express


General Features

The principal functions of Data Express are to:

  • Perform consultation and maintenance functions to define the work area.
  • Schedule batch jobs to run on z/OS. (Data Express for z/OS only.)
  • List application files, with the possibility of applying selection filters with user-defined criteria.
  • Complete and/or selectively access all information available for the selected data store:
    • Details of the record layout
    • Selection of the data classes defined
    • Display (also graphical) of data element contents
  • Perform consultation and maintenance functions to confirm the results of batch phases run on z/OS. (Data Express for z/OS only.)
Parent topic: Front End Guide


Considerations

Be aware of the following points about the Data Builder module for Data Express:

  • To use the Data Express for z/OS solution, you must have the correct license. Refer to the Installation Guide for more details.
  • You do not need to analyze the sources of the programs in the application to use Data Express.
  • Data Express does not perform any operation capable of altering the analyzed data.
Parent topic: Front End Guide


Getting Started with Data Builder

This chapter provides an overview of the Data Builder module within Data Express. You should read it before using the Data Builder module.

  • General characteristics of use
  • Starting the Data Builder module
  • Work environment settings
  • Main functions - phase execution
  • Menus
Parent topic: Front End Guide


General characteristics of use

Data Express for z/OS can be used in either Standard or Client/Server mode, whereas Data Express for Distributed Systems can only be used in Standard mode:

  • Standard – Supports direct access to the data resident on RDBMS from your PC.
  • Client/Server – Supports access to the data resident on RDBMS through the Data Express kbde-Server: a product based on a three-tier architecture where the second tier handles data transfer optimization. Client/Server mode lets you access data over the Internet with the lowest amount of administration necessary and the best utilization of network capacity.
Note: Client/Server mode can use the same functions as Standard mode without you having to install additional software on your PC, such as the DB2 client or the Borland Database Engine (BDE).

For Data Express configuration information and instructions, see Appendix A. Standard Mode and Appendix B. Client / Server Mode.

The following guidelines apply to windows and screen items within Data Express:

  • The main window: [Work with Data Stores] cannot be closed except when exiting the module.
  • Multiple MDI windows may be opened simultaneously.
  • Right-click an item or area to view a list of applicable commands.
  • The list layout in the Grid area can be customized by selecting which columns you want to show or hide, by selecting column width and position, or by ordering columns by increasing or decreasing order. These customizations are automatically saved in the registry so that they are applied the next time the application is run.

Each list can be exported to a file (with several formats available) so that it can be manipulated within a spreadsheet.

Parent topic: Getting Started with Data Builder


Starting the Data Builder module

To start Data Builder:
  1. Click Start > All Programs > Micro Focus Data Express 4.0 > Data Builder- CS
    Note: If you are using Data Express on Windows Vista and User Access Control (UAC) is enabled, you must run Data Builder as Administrator.
  2. Select the required database with which to launch the connection. By default, the highlighted item in the list of available databases is the last database you connected to using the Data Builder module.
    Note: To connect to the database for first time, you need your user ID and password.
  3. Select the required schema.

    The Data Builder - [Work with Data Stores] window appears:

Tip: Once you are connected to a database, you can open and close a database from the [Work with Data Stores] window by clicking File > Open or File > Close.
Parent topic: Getting Started with Data Builder


Work environment settings

This section describes the process of preparing a work environment or workspace. You must specify a workspace to organize and sort data on the basis of a logical model. The creation of at least one machine ID and one company for the work environment is required.

A Data Express detailed work environment comprises the following objects:

  • Machine Identifiers – a primary identifier for your working environment.
  • Companies – a secondary identifier for your working environment.
  • Application Identifiers – applications that use data.
  • Process Identifiers – routines needed by the product to read, analyze, and interpret the data of the processed data stores.
  • Classes – data types that are used to classify the data processed by Data Express.
  • Machine identifier creation
  • Company creation
  • Application identifier creation
  • Default environment parameter settings selection
  • Analysis parameter modification
  • Default copy analysis parameter modification
  • Database interface specification
  • Database access modes
  • Process identifier and database types relationships
Parent topic: Getting Started with Data Builder


Machine identifier creation

You can create a new machine identifier or modify a previously defined one.

One application file (that is managed by Data Express) must be associated with at least one machine identifier. This operation helps you enable a partial analysis of the data stores and a partial display of information while using Data Express.

For more information on the individual fields, see Work with Machine IDs.

Parent topic: Work environment settings


Company creation

You can create a new company or modify a previously defined one.

One application file (that is managed by Data Express) must be associated with at least one company. This operation helps you enable a partial analysis of the data stores and a partial display of information while using Data Express.

For more information on the individual fields, see Work with Companies.

Parent topic: Work environment settings


Application identifier creation

Data Express allows the definition of an Application Identifier. Using this function, you can create a new Application Identifier or modify a previously defined one.

Application IDs can be associated with the data stores of applications managed by Data Express, so as to enable a partial analysis of the data stores and a partial display of information while using Data Express.

For more information on the individual fields, see Work with Application IDs.

Parent topic: Work environment settings


Default environment parameter settings selection

You can set default parameters for the environment, identified by the Machine ID and Company pair. The main parameter, Output Date Format, indicates the format in which dates are displayed in Data Express. For example:

  • DD/MM/YY or DD/MM/YYYY – day/month/year (year with 2 or 4 digits)
  • MM/DD/YY or MM/DD/YYYY – month/day/year (year with 2 or 4 digits)
Note: The Output Date Format parameter is valid for the whole environment and is therefore independent of the Company parameter. To define or change this parameter, click Environment > Work With Default Parameters.
Parent topic: Work environment settings


Analysis parameter modification

Note: This section applies only for Micro Focus Data Express for z/OS.

Data Express includes a function that recognizes the file record image automatically. This function – together with the possibility of analyzing the COBOL and PL/I copies – lets you automatically associate the data store with the copybook describing it.

The applied algorithms are based on statistical analysis of the information. The information included in the data stores is analyzed to indicate the type of data included in each byte.

The analysis algorithms are based on parameters, which can be modified to meet the specific requirements of your database.

The analysis parameters include:

Analysis Option
Indicates the conditions under which a data store must be analyzed. For example:
  • Perform the analysis only if the data store has not been analyzed.
  • Perform the analysis if the data store has been analyzed but not associated.
  • Perform the analysis even if the data store has already been analyzed and associated.
Number of Records
Indicates the number of records to analyze to identify the record image. The larger the number of analyzed records, the more accurate is the recognized record image.
Note: If the number of records to analyze is increased, the time required for the processing increases as well. The default value proposed is normally an acceptable compromise. In many cases, the different data elements included in the data stores may contain null values, such as a low-value, blank value, or all 0 or 9 digits. As a result, we recommend you increase the value of the Number of Records parameter to allow the analysis of a significant number of values.
Sample Data
Indicates whether to perform the analysis of the first n records of the data store (leaving the option blank), or to sample the records to be processed evenly within the data store.
Note: Record sampling increases the time required for processing, but we recommend record sampling for those data stores, which include many non-valued data elements within the first n records.
Copy Association
Indicates whether or not to associate the data store directly with the copybook describing its trace. The association algorithm will only be executed on copybooks analyzed until the copybook association is run. Therefore, we recommend you use this option only when the whole packet of copybooks belonging to the application has been analyzed.
Tolerance
Specifies a percentage of tolerance between the length of the data store layout and that of the copybook describing it.
Note: If you set this parameter to 0 (zero), these length values must coincide. Setting a tolerance to a value different from zero may be useful when ‘FILLER’ fields involving the use of a copybook layout longer than the data store layout are used in the copybook.
DB2 Date External Format
Specifies the format of the DB2 DATE field. This information is required during verification of values outside a validity range or in the identification of special values.
Note: During analysis, default values can be modified temporarily.
Parent topic: Work environment settings


Default copy analysis parameter modification

Note: This section applies only for Micro Focus Data Express for z/OS.

The copybook analysis parameters include:

Analysis Option
Indicates the conditions under which a copybook must be analyzed. For example:
  • Perform the analysis only if the data store has not been analyzed.
  • Perform the analysis if the data store has been analyzed but not associated.
  • Perform the analysis even the data store has already been analyzed and associated.
Maximum Progressive of I/O Area
Specifies the maximum number of incongruent formats an I/O area can describe through the REDEFINES clause (within COBOL copies) or the READDRESSING (within PL/I includes) clause.
COBOL Decimal Separator
Specifies the decimal separator within the COBOL copybook.
Statements beginning in PL/I sources - Statements length in PL/I sources
Specifies the source start column and the length of the source row within the PL/I INCLUDES.
Note: During analysis, default values can be temporarily modified.
Parent topic: Work environment settings


Database interface specification

Note: This section applies only for Micro Focus Data Express for z/OS.

A different default process identifier is loaded for each type of database. This identifier can be replaced if appropriate.

Process identifiers guide Data Express through the processing modes of each database by providing the following information:

Format type
Fixed (FB) or variable (VB) length format.
Access type used
Direct access to the database, access to its sequential unload (for example, DL/I database unload, or DB2 table unload) or access to image-copybook files for DB2 databases.
Length of the sequential unload
Required only for the DL/I sequential unloads.
File identifier within the sequential unload
Required only for the DL/I sequential unloads.
Data record position within the sequential unload
Required only for the DL/I sequential unloads. For all the other types of databases, you can leave the default value.
Data record length within the sequential unload
Required only for the DL/I sequential unloads.
Name of the I/O routine called for the reading of the database.
Parent topic: Work environment settings


Database access modes

This section outlines, in a table, all the database access modes available in the different processing phases.

Note: This section applies only for Micro Focus Data Express for z/OS.

The table is divided into the following main phases:

  • Load Data Store Information
  • Data Store Analysis
  • Data Store Sampling
Data Store Type | Load File Information | Data Store Analysis | Data Store Sampling
DB2 (1) | Direct access / Sequential unload | Direct access / Sequential unload | Direct access / Sequential unload
SEQ | Direct access / Sequential copybook | Direct access / Sequential copybook | Direct access / Sequential copybook
GDG | Direct access / Sequential copybook | Direct access / Sequential copybook | Direct access / Sequential copybook
VSAM | Direct access / Sequential unload (REPRO) | Direct access / Sequential unload (REPRO) | Direct access / Sequential unload (REPRO)
DL/I | Sequential unload | Sequential unload | Sequential unload
Others (2) | Sequential unload / Customized I/O routine | Sequential unload / Customized I/O routine | Sequential unload / Customized I/O routine

(1) Data automatically acquires the column definitions.

(2) Any other database, even from another operating system, can be managed in a sequential format. For databases running with the same system of Data Express, a customized I/O routine can be written and used for a direct access.

Note: The access to the file sequential copybook can use the disk and/or tape unit.
Parent topic: Work environment settings


Process identifier and database types relationships

Note: This section applies only for Micro Focus Data Express for z/OS.

The following table describes process identifiers pre-loaded by Data Express:

Process Identifier | Access Type | Description
ADAUN | Unload File Access (DSNTIAUL) | ADABAS BY UNLOAD
DB2DA | DB Direct Access (output DSNTIAUL) | DB2 DIRECT ACCESS
DB2IY | Image Copy (output UNLOAD) | DB2 BY UNLOAD OF ICOPY FULL
DB2UN | Unload File Access (DSNTIAUL) | DB2 BY UNLOAD
DLIHD | Unload File Access (DSNTIAUL) | DL/I BY UNLOAD HSSR
DLIUN | Unload File Access (DSNTIAUL) | DL/I BY UNLOAD
GENDA | DB Direct Access (output DSNTIAUL) | VSAM,SEQ,GDG DIRECT ACCESS
GENUN | Unload File Access (DSNTIAUL) | VSAM,SEQ,GDG BY UNLOAD
VSMDA | DB Direct Access (output DSNTIAUL) | VSAM DIRECT ACCESS
Parent topic: Work environment settings


Main functions - phase execution

Note: This section applies only for Micro Focus Data Express for z/OS.

The main functions of Data Express can be expressed in phases executed in the following order:

  1. Copybook specification.
  2. Copybook analysis.
  3. Source specification.
  4. Source analysis.
  5. Data store specification.
  6. Data store analysis.
  7. Automatic file or copybook association.

Other phases provided by Data Express are:

  1. Data element sampling.
  2. Data class definitions and association with the data elements.
Note: Many of the phases involve the submission of jobs. After launching a job, you must wait for it to complete before proceeding with the next phase.
Parent topic: Getting Started with Data Builder


Menus

This section describes the menus and common toolbars displayed in the [Work with Data Stores] window.

  • File menu
  • Tools menu
  • Environment menu
  • View menu
  • Windows menu
  • Help menu
  • Context menu
Parent topic: Getting Started with Data Builder


File menu

The File menu contains the following options:
Menu option | Description
Open | Opens a project.
Close | Closes a project.
Close Enterprise View Connection | Disconnects you from the Micro Focus Enterprise View Database. This option is activated after using [image] (view Enterprise View relations).
Database-Schema | Indicates the name of the most recently opened database.
Exit | Terminates the Data Builder module.
Parent topic: Menus


Tools menu

The Tools menu contains the following option:
Menu option | Description
Options | Opens the Option window for setting the help language, security options, Oracle options, filter settings, and confirmation message options.
Parent topic: Menus


Environment menu

The Environment menu contains the following options:
Menu option | Description
Work with Classes | Opens the Work with Classes window for consultation and data entry of classes which can be assigned to a data element of the data store.
Work with Machine IDs | Opens the Work with Machine IDs window for consultation and data entry of identifiers of machines to which the data stores to be analyzed belong.
Work with Companies | Opens the Work with Companies window for consultation and data entry of the companies which use the database to be analyzed.
Work with Application IDs | Opens the Work with Application IDs window for consultation and data entry of the application identifiers.
Work with Process IDs | Opens the Work with Process IDs window for consultation and data entry of the process identifiers.
Work with Default Parameters | Opens the Work with Default Parameters window to change the output date format.
Parent topic: Menus


View menu

The View menu contains the following options:
Menu option | Description
View Data Stores by Structure | Displays the list of data stores on the Structure page.
View Data Stores by Classes | Displays the list of data stores on the Classes page.
View Data Stores by Copybook | Displays the list of data stores on the Assigned to Copy page. This option is listed only in the Data Express for z/OS solution.
View Data Stores by Sampling | Displays the list of data stores on the Sampling page. This option is listed only in the Data Express for z/OS solution.
View Data Stores by Data Changer | Displays the list of data stores on the Data Changer page.
View Data Stores by Register Tables | Displays the list of data stores on the Register Tables page.
Toolbar | Displays or hides the toolbar.
Status bar | Displays or hides the status bar.
Parent topic: Menus


Windows menu

The set of open windows can be organized according to the standard Windows arrangements. The Windows menu contains the following options:
Menu option | Description
Tile Horizontally | Arranges all open windows from top to bottom.
Tile Vertically | Arranges all open windows from left to right.
Cascade | Arranges all open windows so that they do not overlap.
Minimize All | Minimizes all open windows with the exception of the main window, [Work with Data Stores].
Arrange | Arranges all minimized windows.
Parent topic: Menus


Help menu

The Help menu contains the following options:
Menu option | Description
Contents | Opens online Help information for the product in the language selected in the Options window.
Micro Focus SupportLine | Contains commands for connecting to Micro Focus SupportLine for information or technical assistance.
Micro Focus Data Express - Home Page | Connects to the Micro Focus Data Express Web site.
About | Displays production information about the Data Builder module.
Parent topic: Menus


Context menu

This section describes the options displayed in context (pop-up) menus.

  • Column sorting
  • Grid functions
Parent topic: Menus


Column sorting

A context menu to display a list of sort options can be displayed by right-clicking a column header.
Note: All changes to the grid layout are saved in the Windows system registry. For example, groups , column width attributes, column order attributes , and column font and color information. However, filtering and sorting preferences are not saved.

The column sorting context menu contains the following options:

Menu option | Description
Sort Descending | Sorts the list contents in a descending order. The same result can be obtained by simply clicking the column heading.
Sort Ascending | Sorts the list contents in an ascending order. The same result can be obtained by simply clicking the column heading.
Group by This Column | Groups the list according to the contents of the selected column if the Group Panel has been enabled.
Remove This Column | Deletes the selected column.
Column Selector | Displays a Customize window that contains a list of columns that have been removed previously. A column can be restored by performing a drag-and-drop operation to the desired position in the list. A column can be removed from a list by performing a drag-and-drop operation from the list to the Customize window.
Alignment | Aligns the content of the columns to the center, right, or left as specified by the user.
Best Fit | Resizes the selected column to allow the content of each record to be displayed. The same result can be obtained by double-clicking the column's external margin.
Pick Color | Allows the color of the selected column to be changed.
Pick Font | Allows the font, font style, size, color, and script to be changed.
Best Fit (All Columns) | Resizes all the columns from the list to allow the content of each record to be displayed.
Notes:
  • Columns can also be sorted by clicking or performing a drag-and-drop operation on the column heading.
  • Information displayed can be reduced by clicking a column heading, then typing the first letter of the items you are interested in.
Parent topic: Context menu


Grid functions

A context menu to display a list of functions for a grid can be displayed by right-clicking any place on the grid.
Note: All changes to the grid layout are saved in the Windows registry. For example, groups , column width attributes, column order attributes , and column font and color information . However, filtering and sorting preferences are not saved.

The grid functions context menu contains the following options:

Menu option | Description
Column Selector | Displays a Customize window that contains a list of columns that have been removed previously. A column can be restored by performing a drag-and-drop operation to the desired position in the list. A column can be removed from a list by performing a drag-and-drop operation from the list to the Customize window.
Best Fit (All Columns) | Resizes all the columns from the list to allow the content of each record to be displayed.
Export to File Exports the information to a file using one of the following compatible formats:
  • comma separated value (.csv)
  • Hyper Text Markup Language (.htm)
  • text (.txt)
  • Excel (.xls)
  • Extensible Markup Language (.xml)
Show Summary Footer Displays the summary footer beneath the grid. By selecting a row and right-clicking the summary footer, a context menu displaying arithmetic operations you can perform on the selected row appears.

Note: Arithmetic operations are only possible on numeric contents.

Possible arithmetic operations include:

Summary

Displays the sum of all quantities in the column. This option is active only for columns with integer or percentage values; if the value of the cells in a specific column are represented as a percentage their sum does not always equal 100%.

Minimum
Displays the minimum value of all values in the column. The option is active only for columns with numerical values.
Maximum
Displays the maximum value of all values in the column. The option is active only for columns with numerical values.
Count
Displays the number of elements in a column.
Average
Displays the arithmetic mean of all values in the column. The option is active only for columns with numerical values.
None
Specifies using no arithmetic operation on a column.
Show Group Panel Displays the column name by which data is grouped above the grid. By dragging the column headers into this window, a group hierarchy can be defined.
Note: If data is being grouped by column, a footer panel is displayed beneath the grid for each group.
Filter Active If enabled, lets you choose data to be displayed on the grid: a downward arrow icon will appear in the column headers.

Click an arrow to choose the data to be displayed on the grid. This function lets you generate a personal filter.

Note: The time taken to display the window increases with the number of records to be loaded.
Parent topic: Context menu


Main Window - Work with Data Stores

After starting Data Builder, the main window, [Work with Data Stores] appears.

This window provides a list of data stores cataloged by Data Express using different sets of search criteria, including:

  • Machine identifiers
  • Company names
  • Application identifiers
  • Data store types
  • Data stores with assigned classes (Multiple class Selection)
  • Data stores assigned to copybook (Only applicable for Data Express for z/OS)
  • Data stores not assigned to copybook (Only applicable for Data Express for z/OS)
  • Data stores to be sampled
  • Data stores sampled
  • Data stores with sampling inactivated
  • Data stores with sampling activated
  • Data stores with Data Changer activated
  • Data stores with Data Changer inactivated
  • Data stores set as Register Table
  • Data stores without Register Table attribute
  • You can select a tab to view data stores using a specific set of criteria as listed above. Each tab is associated with different grouping criteria.

The list of displayed data stores can be further restricted by applying filters. Filters can be selected from the context menu displayed by right-clicking the List of Data Stores grid, or by selecting the specific icons displayed on the left-hand side of the screen.

In the Data Express for z/OS, the [Work with Data Stores] window comprises six tabular views:
  • Structure
  • Classes
  • Assigned to Copy
  • Sampling
  • Data Change
  • Register Tables

In the Data Express for Distributed Systems, the [Work with Data Stores] window comprises five tabular views:

  • Structure
  • Classes
  • Sampling
  • Data Changer

Note: Double-click a data store name in the grid to view detailed information.
  • Window pages
  • Column headings in the grid area
  • Available grid operations
Parent topic: Front End Guide


Window pages

This section describes each tabbed page of the [Work with Data Stores] window.
  • Structure page
  • Classes page
  • Assigned to Copy page
  • Sampling page
  • Data Changer page
  • Register Tables page
  • Custom Query page
Parent topic: Main Window - Work with Data Stores


Structure page

Root nodes | Description
Machine ID | Data stores are grouped by machine ID.
Company | Data stores are grouped by company ID.
Appl ID | Data stores are grouped by application ID.
File type | Data stores are grouped by type.

Data store groupings can be saved by right-clicking the left-hand side of the screen to display the context menu, then clicking Add to Favorites.

  • All: Reverts to the initial setting and displays all the catalogued data stores.
  • Favorites: Displays the views you have saved.
Parent topic: Window pages


Classes page

Data stores containing data elements with associated classes.

Root nodes | Description
Multiple Class Selection | By dragging two or more classes into the appropriate window, data stores showing both classes can be selected.

Multiple class selections can be saved by right-clicking the left-hand side of the screen to display the context menu, then clicking Add to Favorites.

  • All: Reverts to the initial setting and displays all the classes created.
  • Favorites: Displays the views you have saved.
Parent topic: Window pages


Assigned to Copy page

Data stores assigned to copybook or data stores not assigned to copybook. (Data Express for z/OS only).

Use the icons to display a list of tables with assigned and non-assigned copybooks from the Assigned to Copy page.

  • Assigned: Data stores with assigned copybooks.
  • Unassigned: Data stores without assigned copybooks.
Parent topic: Window pages


Sampling page

Data stores sampled or not sampled. The Data Sampling function can be used to display the content of the data elements or a sample of the values contained in the data elements with the number of occurrences for each of them. The Data Sampling phase provides the following information:

  • Up to 1000 different values for each data element, with their frequency and the frequency of the 999 ranges defined by the 1000 values (standard sampling).
  • Fingerprints (compressed sampling). This is a visual representation of the statistical distribution, in a compressed format, of the data element content.
  • The concise representation of the content of a data element depends on the nature of the data element:
    • The values of numeric data elements are divided into homogeneous groups differentiated by the value of their first digit and the number of digits they include. This description preserves the general representation of the values distributed in the data element.
    • The values of alphanumeric data elements are organized according to several criteria, including: First character, value length, presence of numeric characters only, presence of alphabetic characters only.

The number of values belonging to each class is stored for each of these criteria, which allows the data element content properties to be preserved. If you want to process the compressed sampling, the minimum and maximum values of each sampled data element can also be calculated.

To access the data sampling function, click the [image] tab or click [image] (View Data Stores by Sampling) on the toolbar.

Icon | Description
Activated | Data stores with data elements that have been sampled.
Deactivated | Data stores that contain no data elements with sampling activated.
Sampled | Data stores that contain sampled data elements.
Not Sampled | Data stores that contain no sampled data elements.

The first two icons can be combined with the second two to give four different combinations.

The following options are accessed by right-clicking a selected data store in the Work with Data Stores area on the Sampling page:

Option | Description
Open Selected Data Store | Displays a window, which shows all the objects associated with the data store (Work with Data Stores area).
Data Store Properties | Displays a window with all the information about the data store.
Delete Selected Data Store | Starts a job and deletes highlighted data stores from the Knowledge Base.
Work with Data Elements of the Selected Data Elements | Displays the data element details window, filtered for the currently selected data store.
Show Synthetic Data Element Contents | Shows the statistical distribution of values in graphical form about the sampling of data elements within the selected data store.
Schedule Jobs | Shows a submenu used to schedule jobs for the selected data store. (Data Express for z/OS only).
Enable/Disable Sampling | Enables and disables the sampling function.
Export to File | Exports the list of displayed data to a file of the chosen type.
Filter Active | Independently of the data store selected, applies selections to the list of data stores.
Parent topic: Window pages


Data Changer page

The Data Changer function is a function of the Data Builder module that enables you to manage the privacy of the data contained in the database of a specified application.

In Data Express for z/OS, you generate a copybook applying data masking logic on the basis of exit routines. Records of the original data store can be omitted in the copybook by specifying filtering user exit.

The Data Changer:

  • Provides a list of the application files having the Data Changer functions applied, which may be filtered according to selection criteria defined by the user.
  • Accesses, completely and/or selectively, all the information available for the selected data store:
    • Details of the record layout.
    • Selection of the data classes defined.
    • Display (also graphical) data element contents.

For more information about the Data Changer, see the Data Masking Guide.

To access the Data Changer, click the [image] tab or click [image] (View Data Stores by Data Changer) on the toolbar.

  • Button bar
  • Column headings
  • Context menu
  • Properties- File
Parent topic: Window pages


Button bar

Use the icons below to display a list of tables from the Data Changer page:
Icon | Description
 | Activated: Data stores with Data Changer activated.
 | Deactivated: Data stores with Data Changer deactivated.
Parent topic: Data Changer page


Column headings

The following list describes the columns listed in the List of data stores grid of the Data Changer page from the Work with Data Stores area.

Note: The order and dimensions of the columns are saved by the application automatically so that they are retained for the next time the application is run.
Data Store Name
Name of the data store.
Data Store Type
Identifies the data store type: DB2, DL/I, Sequential (SEQ), VSAM, GDG ODBC, or ORA. Data stores are grouped by type.
Machine ID
Identifier of the machine. Data stores are grouped by machine ID.
Company
Company name. Data stores are grouped by company ID.
Record format
Name of the record format.
Application ID
Identifier (if it exists) of the application area the data store belongs to. Data stores are grouped by application ID.
Number of Records
Number of records contained in the data store.
Record Length
Length in bytes of the record layout.
Date of Migration
Date on which the data store was migrated.
Number of Analyzed Records
vNumber of records analyzed in the data store.
Unload Input Data Store Type
Type of the Unload Input data store.
Unload Input File Name
Name of the Unload Input data store.
Unload Input File Version
Version of the Unload Input store.
Format Selector
Exit-routine for recognizing formats.
Select Program
Selection program of the individual record.
Unload Output Data Store Type
Type of the Unload Output data store.
Unload Output Data Store Name
Name of the Unload Output data store.
Unload Output Data Store Version
Version of the Unload Output data store.
Write Program for Output
Write program of the Output data store.
Associated Copybook PDS
Name of the PDS, which contains the copybook associated with the file. (Only applicable for Data Express for z/OS.)
Assigned Copybook Name
Name of the copybook associated with the data store. (Only applicable for Data Express for z/OS.)
IO area Name
Name of the area (level 01) within the copybook associated with the data store. (Only applicable for Data Express for z/OS.)
IO area Prg.
Progressive number of the area within the copybook associated with the data store.(Only applicable for Data Express for z/OS.)
Copy Association Provider
Identifies who supplied the association information; PGM is information obtained from batch processing, USR is information supplied by the user. (Only applicable for Data Express for z/OS.)
Description
Any text associated with the data store.
Data Store Nickname
Internal name given to the data store.
Parent topic: Data Changer page


Context menu

This section describes options you can access by right-clicking a selected data store in the Work with Data Stores area from the Data Changer page.
Open the Selected Data Store
Displays a window that shows all the objects associated with the data store in a Work with Data Store window.
Data Store Properties
Displays a window with all the information about the data store. See Properties- File.
Delete Selected Data Store
Deletes information for the highlighted data stores from the Knowledge Base.
Work with Data Elements of the Selected Data Store
Displays the data elements details window, filtered for the currently selected data store.
Show Synthetic Data Element Contents
Shows information in graphical form about the sampling of data elements within the selected data store.
Schedule Jobs
Shows a submenu used to schedule jobs for the selected data store. (Data Express for z/OS only.)
Export to File
Exports the list of displayed data to a data store of the chosen type.
Filter Active
Independently of the data store selected, applies selections to the list of data stores.
Parent topic: Data Changer page


Properties- File

The Properties - File window is a tabbed view that displays all attributes for a selected data store.

To access the Properties - File window, display the Data Changer context menu and click Data Store Properties. An example of the Properties - File window is shown below:

The fields on the Changer Input page are:

Format selector
Exit routine for recognizing formats.
Select program
Selection program of the individual record.
Process ID
Process identifier.
Unload data store type
Type of the Unload Input data store.
Unload data store name
Name of the Unload Input data store.
Unload data store version
Version of the Unload Input data store.

The fields listed on the Making Output page are:

Write program
Write program of the output file.
Unload data store type
Type of the Unload Output data store.
Unload data store name
Name of the Unload Output data store.
Unload data store version
Version of the Unload Output data store.
Parent topic: Data Changer page


Register Tables page

The Register Tables page shows the environment files divided into two subsets: Register Table files and Other Table files.

When data stores are loaded into the environment, they are initially all stored in theOther Table category. If, during the sampling phase, a data store contains at least one data element containing all different values, the tool considers this data store to be a register table file, and stores it in the Register Table subset.

You can use the specially provided commands to modify this subdivision at any time. The splitting up into Register Table files and Other Table files has mainly been introduced for the Data Subset Extraction (DSE) module.

The DSE function helps you create methodologies for the automatic generation of complete or reduced test environments starting from an existing environment and on the basis of selection criteria entered by the user. Additionally, the function lets you apply data masking logic during the generation of new environments, including:

  • Integral test environment
  • Reduced test environment
  • Migrated environment

For further information on the Data Subset Extraction function, refer to the Data Subset Extraction Guide.

To access this function, click the [image] tab or click [image] (View Data Stores by Register Tables) on the toolbar

  • Button bar
  • Context menu
Parent topic: Window pages


Button bar

Use the icons below to display a list of data stores from the Register Tables page:
Icon | Description
 | Register Tables: Data stores set as register tables.
 | Other Tables: Data stores that do not have the Register Tables flag activated.
 | Automatically Provided: Data stores that have been automatically assigned this characteristic during the sampling phase. (Data Express for z/OS only).
 | Provided By The User: Data stores that have been manually assigned this characteristic.

The first two icons can be combined with the second two to give four different combinations.

Parent topic: Register Tables page


Context menu

This section describes commands listed by right-clicking a selected data store in the Work with Data Stores area from the Register Tables page.

Open the Selected Data Store
Displays a window showing all the objects associated with the data store (Work with Data Store window).
Data Store Properties
Displays a window with all the information about the data store.
Delete Selected Data Store
Deletes information for the highlighted data store from the Knowledge Base.
Work with Data Elements of the Selected Data Store
Displays the data element details window filtered for the currently selected data store.
Show Synthetic Data Elements Contents
Shows information in graphical form about the sampling of data elements within the selected data store.
Schedule Jobs
Shows a submenu used to schedule jobs for the selected data store. (Data Express for z/OS only.)
Set as Register Table
Manually sets the data store as Register Table.
Set as Not Register Table
Manually sets the data store as Other Table.
Export to File
Exports the list of displayed data to a file of the chosen type.
Filter Active
Independently of the selected data store, applies selections to the list of data stores.
Parent topic: Register Tables page


Custom Query page

The Custom Query tab appears only after a custom query has been created. To create a custom query, see New Custom Query Wizard.
Parent topic: Window pages


Column headings in the grid area

The Grid area is on the right-hand side of each tab. The following list describes the column headings in the Grid area for data stores that have been loaded in the Data Express Knowledge Base:
Machine ID
Identifier of the machine.
Company
Name of company.
Data Store Name
Name of the data store.
Data Store type
Type of data store. For example, DB2, VSAM, DL/I, GDG, SEQ, GDG ODBC, or ORA.
Record format
Name of the record format.
Application ID
Identifier (if it exists) of the application area the data store belongs to.
Associated Copybook PDS
Name of the PDS that contains the copybook associated with the data store. (Applicable only for Data Express for z/OS.)
Assigned Copybook Name
Name of the copybook associated with the data store. (Applicable only for Data Express for z/OS.)
IOarea Name
Name of the area (level 01) within the copybook associated with the data store. (Applicable only for Data Express for z/OS.)
IOarea Progr
Progressive number of the area within the copybook associated with the data store. (Applicable only for Data Express for z/OS.)
Description
Brief description of the data store.
Copy Association Provider
Supplier of the association information. PGM is information obtained from batch processing, USR is information supplied by the user.
Number of Records
Number of records contained in the data store.
Record Length
Length in bytes of the record layout.
Date of Last Calculation
Date of the last calculation run on the data store.
Date of Last Sample
Date of the last calculation of the distribution of values of the data elements of the data store involved in that process.
Date of Analysis
Date of the analysis carried out on the data store.
Number of Analyzed Records
Number of records analyzed in the data store.
File Nickname
Internal name given to the data store.
Sampling
Indicator of whether the Sampling function has been enabled.
Parent topic: Main Window - Work with Data Stores


Available grid operations

Right-click a table name in the grid to view a list of available commands as described below:
Note: Not all commands are available on all pages.
Open Selected Data Store
Displays the Work with Data Store window that shows all the objects associated with the data store.
Data Store Properties
Displays a window listing data store information.
Assign an Application ID
Displays the Assign an application ID window where you can assign an application ID to a single data store or assign multiple application IDs to multiple data stores. Not available from the Data Changer page.
Delete Selected Data Store
Deletes information for the selected data store from the Knowledge Base.
Work with Data Elements of the Selected Data Store
Displays the Work with Data Elements window that shows data element details, filtered for the selected data store.
Show Synthetic Data Element Contents
Displays information in graphical form about the sampling of data elements within the selected data store.
Schedule Jobs
Displays a submenu used to schedule jobs on the selected data store. (Data Express for z/OS only.)
Enable Sampling or Disable Sampling
Toggles the Sampling function on and off.
Set as Register Tables
Manually sets the data store as Register Table. This command is only available from the context menu accessible from the Register Table page.
Reset as Register Table
Manually sets the data store as Other Table. This command is only available from the context menu accessible from the Register Tables page.
Parent topic: Main Window - Work with Data Stores


Work with Data Store Window

The Work with Data Store window provides information about the selected data store and all the objects related to it. The information related to a data store includes:

  • The data elements in the data store.
  • The copybooks that describe the data store. (Applicable only for Data Express for z/OS.)
  • The classes assigned to the data elements.
  • The distribution of values in the data elements (spectrum).
  • The information about the data stores that have the Data Masking routines applied.
  • To access the Work with Data Store window, right-click a table in the Grid area (on the appropriate page), and click Open Selected Data Store.
  • Window contents
Parent topic: Front End Guide


Window contents

This section describes the fields and items listed in the Work with Data Store area as shown below:

The left pane contains a tree structure containing nodes with the available information about the opened data store. The right-hand part of the screen shows a list of data linked to the selected node.

Note: The Parent Segment field appears for DL/I files and is an indicator of whether the data store is a parent segment. (Data Express for z/OS only.)
  • Node pane
  • Grid
  • Context menu
  • Data element properties
Parent topic: Work with Data Store Window


Node pane

The root nodes are:

Data Elements
Shows a detailed list of all the data elements of the data store, and shows summary information in the data elements listed above the grid. This node does not contain sub-level nodes.
Associated Copybook
Shows a detailed list of the copybooks that are compatible with the data store. This node contains sub-level nodes, which are the copybooks. When a copybook is selected, a list of fields is displayed, and summary information is displayed in the fields above the grid. (Data Express for z/OS only.)
Assigned Classes
Shows a detailed list of all the data elements of the data store associated with the class. This node contains sub-level nodes, which are the associated classes. When a class is selected, a list of fields is displayed, and summary information is displayed in the data elements above the grid.
Sampling
Shows a detailed list of all data elements of the data store for which the sampling of the values has been activated, and those for which the sampling of the values has been calculated at least once. This node contains sub-level nodes, which populate the fields listed above the grid when selected.
Parent topic: Window contents


Grid

The columns displayed in the Grid area depend on the node selected in the Node pane.
  • Data Elements columns
  • Associated Copybook columns
  • Associated Copybook - CopybookName columns
  • Assigned Classes columns
  • Sampling columns
  • Sampling – DataElementName columns
Parent topic: Window contents


Data Elements columns

The following list describes the columns listed by selecting the Data Elements node:
Name
Name of the data element within the data store.
Displacement
Offset of the data element.
Physical Displacement
Physical displacement of a DB2 field.
Note: Only DB2 fields use a Physical Displacement value.
Type
Data type of the data element. For example, Numeric or Alphanumeric.
Nature
Nature of the data elements, for example, Alphabetical.
Integer
Number of integers within a numeric data element.
Length
Length of the data element in bytes.
Decimal
Number of digits included to the right of the decimal point.
Assigned Class
Description of the associated class.
Estimated Class
Description of the most likely class to associate with the data element.
Minimum Value
Minimum value for the data element (result of data sampling).
Maximum Value
Maximum value for the data element (result of data sampling).
No. of Different Values
Number of different values that the data element can take on. This is the principal result of sampling the data store.
Parent topic: Grid


Associated Copybook columns

The following list describes the columns listed by selecting the Associated Copybook node. The Associated Copybook node only applies to the Data Express for z/OS solution.
Machine ID
Identifier of the machine.
Company
Name of the company.
Copybook name
Name of the copybook (associated/compatible)
Copy PDS
Partitioned data set of the copybook.
I/O Area Name
Name of the associated area within the copybook (equivalent to level 01 of COBOL copybooks or of PL/I includes).
I/O Area Progressive
Progressive area number. If greater than 0 (zero), indicates the existence of incongruent redefinitions of the area.
Deleted Flags
Logical deletion of the data store/copybook association
Record Length
Length of the record in bytes.
Number of Matches
Number of congruent bytes between the record layout of the data store and the format of the copybook.
Information Provider
Origin of the information; PGM is supplied by batch process, USR is supplied by user.
Associated Copy
An indicator of whether a copybook is associated with the data store (Yes, No).
Parent topic: Grid


Associated Copybook - CopybookName columns

Associated Copybook - CopybookName is a subnode of the Associated Copybook node.
Note: The Associated Copybook - CopybookName node only applies to the Data Express for z/OS solution.

The following list describes the columns listed by selecting the Associated Copybook - CopybookName node:

Name
Name of the field within the copybook.
Displacement
Position of the field within the layout of the copybook.
Level
Level of the field within the COBOL copybook or PL/I include.
Type
Type of copybook field. For example, Numeric or Alphanumeric.
Length
Length of copybook field in bytes.
Integer
Number of integer digits in the copybook field.
Decimal
Number of digits included to the right of the decimal point.
Sign
An indicator of whether the field is signed (Yes, No).
OCCURS Level
An indicator if the field is defined with recurrences.
Group
An indicator whether the field is of a group (Yes, No).
Parent topic: Grid


Assigned Classes columns

The following list describes the columns listed by selecting the Assigned Classes node:
Name
Name of the data element within the data store.
Displacement
The offset of the data element.
Physical Displacement
The physical displacement of a DB2 field.
Note: Only DB2 fields use a Physical Displacement value.
Type
Data type of the data element. For example, Numeric or Alphanumeric.
Nature
Nature of the data element, for example, Alphabetical.
Length
Length of the data element in bytes.
Integer
Number of integers within a numeric data element.
Decimal
Number of digits included to the right of the decimal point.
Assigned Class
Description of the associated class.
Estimated Class
Description of the most likely class to associate with the data element.
Minimum Value
Minimum value for the data element (result of data sampling). (Applicable only for Data Express for z/OS.)
Maximum Value
Maximum value for the data element (result of data sampling). (Applicable only for Data Express for z/OS.)
No. of Different Values
Number of different values that the data element can take on. This is the principal result of sampling the data element. (Applicable only for Data Express for z/OS.)
Parent topic: Grid


Sampling columns

The following list describes the columns listed by selecting the Sampling node. The Sampling node only applies to the Data Express for z/OS solution.
Name
Name of the data element within the data store.
Displacement
The offset of the data element.
Physical Displacement
The physical displacement of a DB2 field.
Note: Only DB2 fields use a Physical Displacement value.
Type
Data type of the data element. For example, Numeric or Alphanumeric.
Nature
Nature of the data element. For example, Alphabetical.
Length
Length of the data element in bytes.
Integer
Number of integers within a numeric data element.
Decimal
Number of digits included to the right of the decimal point.
Assigned Class
Description of the associated class.
Estimated Class
Description of the most likely class to associate with the data element.
Minimum Value
Minimum value for the data element (result of data sampling).
Maximum Value
Maximum value for the data element (result of data sampling).
No. of Different Values
Number of different values that the data element can take on. This is the principal result of sampling the data store.
Parent topic: Grid


Sampling – DataElementName columns

Sampling – DataElementName is a subnode of the Sampling node.

The following list describes the columns listed by selecting the Sampling – DataElementName node:

Field Value
Value assumed by the data element.
Occurrence No
Number of times the value in Field Value has been found.
Percentage
Percentage of the value over all the occurrences.
In-Range Occurrences
In the complete sample, up to 1000 different values are recorded for each data element. Each new value is considered within the range defined by the values immediately above and below. This field indicates the occurrence within such ranges.
Recalculation Date
Date of the last sampling.
Parent topic: Grid


Context menu

This section outlines the commands displayed in the context menu when selecting a data element in the Work with Data Store area, and then right-clicking to display the context menu.
Note: All changes of the grid layout are saved in the Windows system registry. For example, groups, column width attributes, column order attributes, and column font and color information. However, filtering and sorting preferences are not saved.

The data element context menu contains the following commands:

Zoom of the Synthetic Data Element Content
Displays the window showing the Zoom of the distribution of the data element values (provided that there are values sampled for the data element).
Data Element Properties
Displays a window with summary information about the data store.
Show In Sampling
Displays a window showing the sampling results.
Set as Register Table Data Element
Sets the selected data element as a Register Table data element.
Reset as Register Table Data Element
Resets the selected data element as a Register Table data element.
Add to Combined Data Element
Displays the Add to Combined Data Element window where you can define a new data element (which is a combination of multiple data elements), after multiple data elements have been selected.
Parent topic: Window contents


Data element properties

The Data Element Properties window displays all attributes for a selected data element.

To access the Data Element Properties window, display the context menu from the main window [Work with Data Element] for a selected data element, and click Data Element Properties.

The following list describes the fields listed in the Data Element Properties window:

Name
Name of the data element within the data store.
Description
Data store description.
Type
Type of data element.
Nature
Nature of the data element. For example, Numeric or Alphanumeric.
Displacement
Field offset at the beginning of the I/O area. (Data Express for z/OS only)
Length
Length of data element in bytes.
Integer
Number of integer digits present.
Decimal
Number of decimal digits present.
Register Table Data Element
Indicator of whether the data element is unique in the data store (Yes, No).
Data Element Sign
Indicator of whether the data element is considered as signed or not signed.
Variable Length
Indicator of whether the length of the data element is variable.
Justified
Indicator of whether the data element is justified.
Nullable Data Element
Indicator of whether the data element is set to NULL.

The buttons at the bottom of the fields in the Data Element Properties window are:

Change
Initializes fields for insertion.
OK
Saves any changes that have not been applied and closes the window.
Cancel
Cancels any changes to the field properties and closes the window.
Parent topic: Window contents


Assign an Application ID Window

You can assign an application ID to a single data store or assign multiple application IDs to multiple data stores.

Note: You can call the Assign an Application ID window from all pages on the [Work with Data Stores] main window except the Data Changer page.
  1. From the [Work with Data Stores] main window, select one or more data stores on the grid; then right-click and select Assign an Application ID from the context menu.
  2. Do either of the following:
    To assign a single application ID to all selected data stores
    From the Select an application ID drop-down list, select an application ID; then click Apply.
    To assign multiple application IDs to multiple files
    1. Click Assign Multiple.
    2. Using the drop-down list in the Application ID column for each corresponding data store, select an application ID to associate with each data store.
    3. When finished, click Apply.
  • Window contents - Assign Single
  • Window contents - Assign Multiple
Parent topic: Front End Guide


Window contents - Assign Single

Describes the fields and items that appear on the Assign an Application ID window for assigning a single application ID to one or more data stores.

Assign an Application ID - Assign Single
Select an application ID
Lists all available application IDs that can be applied to the selected data store or data stores.
Assign Multiple
Opens the Assign an Application ID window for assigning multiple IDs to multiple selected data stores.
Apply
Applies the selected application ID to the selected data store or data stores.
Parent topic: Assign an Application ID Window


Window contents - Assign Multiple

Describes the fields and items that appear on the Assign an Application ID window for assigning multiple application IDs to multiple data stores. The following shows four example data stores:

Assign an Application ID - Assign Multiple
File Name
Name of the data store.
File type
Type of data store. For example, DB2, VSAM, DL/I, GDG, SEQ, GDG ODBC, or ORA.
Application ID
Identifier of the application area to assign to the corresponding data store.
Assign Multiple
Opens the Assign an Application ID window for assigning a single application ID to the selected data stores.
Apply
Applies the selected application IDs to their corresponding data stores.
Parent topic: Assign an Application ID Window


Add to Combined Data Element Window

The Add to Combined Data Element window displays information about the combined data element you are managing and the data store layout data element that comprise the combination. This window lets you display, modify, or delete data element combination information.

A combined data element is essentially a “dummy” data element that you create to manage a combination of different data elements that belong to the same data store.

To access the Add to Combined Data Element window, right-click one or more fields in the Grid area of the Work with Data Store area and click Add to Combined Data Element.

An example of the Add to Combined Data Element window is shown below:

Note: For more information about accessing the Work with Data Element area and the Data Element context menu, see Work with Data Store Window.

For data subsetting only, the Advanced features window enables you to modify the contents of a data element in a combined data element. For example, you can add prefix or suffix content to the data element using character or hexadecimal values. In addition, you can substring a part of the data element, or supply a routine that decides what the content of the data element should be (similar to a masking routine).

Note:
  • If you use a routine to define the data element contents, you cannot use other options.
  • All other options can be used together, if required.
  • Advanced features can only be used with data subsetting output classes, and cannot be used with input classes.
  • Advanced features are available only with z/OS data stores.

An example of the Advanced features window is shown below:

Advanced features window
  • Combined data element requirements and limitations
  • Available operations
Parent topic: Front End Guide


Combined data element requirements and limitations

Before data elements can be encompassed into a combined data element, they must be checked for correctness. Each data element must meet the following requirements:

  • The data element is part of the data store that is being processed.
  • The data element is not a combined data element.
  • The data element is not a BIT data element.

There are limitations to the management of combined data elements as follows:

  • A previously defined combined data element cannot be inserted in the list of data elements.
  • There is a 230-digit limit to the length of the fields for sampling and subsetting operations. If you want to perform these operations on longer combined data elements, you must develop some exit routines. (Data Express for z/OS only.)
  • A combined data element cannot be masked unless it is an MVS data element and its elementary data elements are consecutive, emulating the masking of a group COBOL field. (Data Express for z/OS only.)
Parent topic: Add to Combined Data Element Window


Available operations

The buttons at the bottom of the Add to Combined Data Element window are:

Refresh
Refreshes the information. If another user defines a combined data element from a different executable in the same database, you can see the changes.
Apply
Confirms the changes made to the combined data element. A newly created combined data element appears as the last field in the List of Data Elements grid.
Delete
Deletes all information for the selected combined data element.
Cancel
Cancels any changes applied to the selected data element. Any actions confirmed with the Apply button cannot be restored with Cancel but must be entered and confirmed again.
Drag-and-drop operations
Data elements listed in the List of Data Elements grid can be added to the List of Data Elements grid by using a drag-and-drop operation.
Parent topic: Add to Combined Data Element Window


Work with Classes

The Work with Classes area lets you manage classes by categorizing data element with homogeneous characteristics and by writing generalized routines to check data.

A class identifies a user-defined group of data element with homogeneous characteristics. Typical classes of data can be quantities, prices, client code, totals, or currency flags. Each data element can be cataloged as a member of a class.

Note: The Data Builder module provides some predefined classes, which you can use but not modify.

Classes are grouped into sets called Super Classes. A Super Class gathers several different classes in a single set. During the class automatic assignment phase, this type of classification enables you to restrict the search for classes to a subset of all the defined classes.

The set of Super Classes is as follows:

  • Numbers (N)
  • Codes (C)
  • Date (D)
  • Complemented Date (9)
  • Descriptions (X)
  • Privacy (P)
  • Other (O)
  • None
Note: The Super Class Other is used for new user classes that may not fall into any of the previous Super Classes. Conversely, the Super Class None is used for classes that do not belong to a Super Class.

Date classes are defined by four available formats so you can run a formal check on the validity of the date fields. For more information about the format of dates, see Getting Started With Data Builder

  • Work with Classes window
Parent topic: Front End Guide


Work with Classes window

The Work with Classes window appears after clicking [image] (Work with Classes) from the main window.

The upper part of the window contains a list of classes, which is continuously updated according to the functions performed. Below the list is a group of objects that map to all the data elements of the selected class.

The following table describes the columns and fields listed in the Work with Classes area.

Note: If the corresponding Min and Max fields maintain their default values, the data range is ignored and only the Data type value is considered.
Field/Column Name | Description | Additional Information/Requirements
Class ID | The Class identifier, which is a unique key for the file that records the classes. | Mandatory field that is only editable during class creation. Note: Class IDs 9990-9999 are reserved for use by the Data Subset Extraction module to implement OR conditions during subsetting.
Super Class | The Super Class that has been associated with the class. | If the Super Class value has been set to a space character, it indicates that no Super Class has been associated with the data element.
Name | The name of the class. | Mandatory field.
Charcode | The character identifier of the class. | Optional field.
Description | A brief description of the class. The information in this column matches the information in the Full Description field. | Mandatory field.
Highlight | The primary class flag. | Optional field.
Edit Mask | The class editing mask. | Optional field.
Changer Program | The program which will be used when the class is assigned as a changer for a selected field. | Optional field.
Data type | Allowed data type for the class. | Mandatory field. This attribute is relevant only if you use the associating data stores/copybooks automatically function. It is also used by Data Subset Extraction.
Min Length | Allowed minimum length range for the class. The default value is 0. | This attribute is relevant only if you use the associating data stores/copybooks automatically function. Note: A check on the length is only performed on non-numeric classes.
Min Int | Allowed minimum number of integer digits for the class. The default value is 0. | This attribute is relevant only if you use the associating data stores/copybooks automatically function. Note: A check on the length is only performed on non-numeric classes.
Min Dec | Allowed minimum number of decimal digits for the class. The default value is 99. | This attribute is relevant only if you use the associating data stores/copybooks automatically function. Note: A check on the length is only performed on non-numeric classes.
Max Length | Allowed maximum length range for the class. The default value is 0. | This attribute is relevant only if you use the associating data stores/copybooks automatically function. Note: A check on the length is only performed on non-numeric classes.
Max Int | Allowed maximum number of integer digits for the class. The default value is 0. | The Min Int and Max Int values must be specified if you receive the message: “Error! Reference source not found.” Note: If the class is non-numeric, this field is ignored.
Max Dec | Allowed maximum number of decimal digits for the class. The default value is 99. | This attribute is relevant only if you use the associating data stores/copybooks automatically function. Note: A check on the length is only performed on non-numeric classes.
Name contains | Sequence of characters that data elements associated with the class must contain. | Optional field.
Name does not contain | Sequence of characters that data elements associated with the class must exclude. | Optional field.
Refresh button | Refreshes the class information. | If another user creates a class from a different executable in the same database, you can see the changes.
Apply button | Confirms the changes made to the class.
New button | Initializes fields for insertion when creating a new class.
Delete button | Deletes the selected class. Before a class can be deleted, a series of referential integrity checks are carried out which, if not passed, generate an error message and block the deletion.
Cancel button | Cancels any changes applied to the selected class. Any actions confirmed with Apply cannot be restored with Cancel but must be entered and confirmed again.
OK button | Saves any changes that have not been applied and closes the window.
Parent topic: Work with Classes


Work with Machine IDs

The Work with Machine IDs area lets you perform consultation and data entry (insert, edit, and delete) operations on classes. The insertion and edit data entry operations are stored. These operations carry out a series of formal controls on the data element types. When deleting, however, checks are made on referential integrity on the data stores, which depend on the machine identifiers.

To access the Work with Machine IDs window, click [image] (Work with Machine IDs) from the main window.

  • Work with Machine IDs window
Parent topic: Front End Guide


Work with Machine IDs window

This section describes the columns and fields listed in the Work with Machine IDs window.

The upper part of the window contains a list of machine identifiers, which is continuously updated according to the functions performed. Below the list is a group of objects that map to all the data elements of the selected machine identifier. The following list describes the controls listed in the Machine IDs window:

Control | Description
Field:
Machine ID | Identifier of the machine. This field uniquely identifies the element and cannot be changed.
Description | Description of the machine identifier.
Model | Model of the machine.
Processor | Processor in the machine.
Insert Date | Date when the new machine ID was created.
Insert Time | Time when the new machine ID was created.
Button:
Refresh | The machine ID information. If another user creates a machine ID from a different executable in the same database, you can see the changes.
Apply | Confirms the changes made to the machine ID. Before changes are applied, a series of checks on data elements are performed; if the checks fail, the update is blocked.
New | Initializes fields for insertion when creating a new machine ID.
Delete | Deletes the selected machine ID. Before a machine ID can be deleted, a series of referential integrity checks are carried out which, if not passed, generate an error message and block the deletion. Note: Before deleting a machine ID, any company identifiers contained within it must be deleted in the Work with Companies area. For more information, see Work with Companies.
Cancel | Cancels any changes applied to the selected machine identifier. Any actions confirmed with Apply cannot be restored with Cancel but must be entered and confirmed again.
OK | Saves any changes that have not been applied and closes the window.
Parent topic: Work with Machine IDs


Work with Companies

The Work with Companies area lets you perform consultation and data entry (insert, edit, and delete) operations on the companies cataloged by Data Express.

The insert and edit data entry operations are stored. These operations carry out a series of formal controls on the data element types. When deleting, however, checks are made on referential integrity on the data stores, which depend on the company identifiers.

A new company ID may be inserted only if an existing machine ID is selected. If a new machine ID is required, it must be created from the Work with Machine IDs area. For more information about the Work with Machine IDs area, see Work with Machine IDs.

To access the Work with Companies window, click [image] (Work with Companies) from the main window.

  • Window contents
  • Available operations
Parent topic: Front End Guide


Window contents

This section describes the columns and fields listed in the Work with Companies window as shown below:

Work with Companies

The upper part of the window contains a list of companies, which is continuously updated according to the functions performed. Below the list is a group of objects that map to all the fields of the selected company.

Machine ID
Identifier of the machine. This field uniquely identifies the element and cannot be changed.
Company Code
Company identifier. This field uniquely identifies the element and cannot be changed.
Description
Description of the company identifier. The information in this column matches the information in the Company Description field.
Insert Date
Date when the new company was created.
Insert Time
Time when the new company was created.
Life Cycle Activated
Indicator of whether the Life Cycle is activated or deactivated.
Backup Information
Information on the corresponding backup company including the Machine ID and Company Code. If Is a Backup Company box is checked, the company is used by the Life Cycle to save the old file information. To remove backup company information, check Clear Company.
Record Limits
Specifies a limit to the number of records generated by Data Subset Extraction. If this value is 0 (zero), normal subsetting rules apply. Otherwise, the value specified is the number of records generated by the subsetting process.
Note: When Record Limits is specified at both the company and method level, Data Express uses the method value.
Parent topic: Work with Companies


Available operations

The buttons at the bottom of the Work with Companies area are:
Refresh
Refreshes the company information. If another user creates a company from a different executable in the same database, you can see the changes.
Apply
Confirms the changes made to the company. Before changes are applied, a series of checks on non-key fields are performed; if the checks fail, the update is blocked.
New
Initializes fields for insertion when creating a new company.
Delete
Deletes the selected company.
Cancel
Cancels any changes applied to the selected company. Any actions confirmed with Apply cannot be restored with Cancel but must be entered and confirmed again.
OK
Applies the changes immediately and closes the window.
Clear Company
Clears the information on the associated backup company.
Analysis Parameters
Launches the Define Analysis Parameters Wizard where copybook analysis parameters can be defined.
Parent topic: Work with Companies


Work with Application IDs

The Work with Application IDs area lets you perform consultation and data entry (insert, edit, and delete) operations on the application IDs cataloged by Data Express.

The insertion and edit data entry operations are stored. These operations carry out a series of formal controls on the data element types. When deleting, however, checks are made on referential integrity on the data stores, which depend on the application identifiers.

To access the Work with Application IDs window, click [image] (Work with Application IDs) from the main window.

  • Window contents
  • Available operations
Parent topic: Front End Guide


Window contents

This section describes the columns and fields listed in the Work with Application IDs area as shown below:

The upper part of the window contains a list of application identifiers, which is continuously updated according to the functions performed. Below the list is a group of objects that map to all the fields of the selected element.

Application Identifier
Identifier of the application.
Description
Description of the application identifier. The information in this column matches the information in the Application ID Description field.
Insert Date
Date when the new application ID was created.
Insert Time
Time when the new application ID was created.
Parent topic: Work with Application IDs


Available operations

The buttons at the bottom of the Work with Application IDs area are:
Refresh
Refreshes the application identifier information. If another user creates an application identifier from a different executable in the same database, you can see the changes.
Apply
Confirms the changes made to the application identifier. Before changes are applied, a series of checks on non-key data elements are performed. If the checks fail, the update is blocked.
Note: Only the Application ID Description field can be modified.
New
Initializes fields for insertion when creating a new application identifier.
Delete
Deletes the selected application identifier. An application identifier can only be deleted if data stores have not been cataloged for it by Data Express.
Cancel
Cancels any modifications made to the selected application identifier. Any actions confirmed with Apply cannot be restored with Cancel but must be entered and confirmed again.
OK
Saves any changes that have not been applied and closes the window.
Parent topic: Work with Application IDs


Work with Routines

The Work with Routines area lets you perform consultation and data entry (insert, edit, and delete) operations on the masking and subsetting routines by Data Express, allowing you to document the masking and subsetting routine used in the project.

To access the Work with Routines window, click Work with Routines (Work with Routines) from the main window.

  • Window contents
  • Available operations
Parent topic: Front End Guide


Window contents

This section describes the columns and fields listed in the Work with Routines area as shown below:
Work with Routines

The upper part of the window contains a list of routines, which is continuously updated according to the functions performed. Below the list is a group of objects that map to all the fields of the selected element.

Routine Name
Name of the routine.
Routine Description
Description of the routine.
Routine Type
Type of the routine (masking or subsetting).
Parent topic: Work with Routines


Available operations

The buttons at the bottom of the Work with Routines area are:
Apply
Confirms the changes made to the routine description.
New
Initializes fields for insertion when creating a new routine.
Delete
Deletes the selected routine. A routine can only be deleted if it is not used by Data Express.
Cancel
Cancels any modifications made to the selected routine. Any actions confirmed with Apply cannot be restored with Cancel but must be entered and confirmed again.
OK
Saves any changes that have not been applied and closes the window.
Parent topic: Work with Routines


Work with Process IDs

The Work with Process IDs area lets you perform consultation and data entry (insert, edit, and delete) operations on the process IDs cataloged by Data Express.

The Work with Process IDs window is available only in the Data Express for z/OS solution.

The insertion and edit data entry operations are stored. These operations carry out a series of formal controls on the data element types. When deleting, however, checks are made on referential integrity on the data stores, which depend on the process identifiers.

To access the Work with Process IDs window, click [image] (Work with Process IDs) from the main window.

  • Window contents
  • Available operations
Parent topic: Front End Guide


Window contents

This section describes the columns and fields listed in the Work with Process IDs area as shown below:

The upper part of the window contains a list of process identifiers, which is continuously updated according to the functions performed. Below the list is a group of objects that map to all the fields of the selected element.

The following describes the general fields listed in the Work with Process IDs area:

Field | Description
Process identifier | Identifier of process. This field uniquely identifies the element and cannot be changed.
Access type Access type to be used:
  • Direct access to database, output in DSNTIAUL format.
  • Direct access to database, output in UNLOAD format.
  • Access to sequential unload in DSNTIAUL format.
  • Access to sequential unload in UNLOAD format.
  • Access to sequential unload in UNLOAD by REORG format.
  • Access to data set of full image-copy, output in DSNTIAUL format.
  • Access to data set of full image-copy, output in UNLOAD format.
  • Access to CA variable sequential unload, output in DSNTIAUL format.
  • Access to CA comma delimited sequential unload, output in DSNTIAUL format.
  • Access to CA external sequential unload, output in DSNTIAUL format.
  • Access to CA fixed sequential unload, output in DSNTIAUL format.
Record format | Format of the record (FB for fixed length format, VB for variable length format).
Description | Description of the process identifier.
I/O program name | I/O routine called for the reading of the database.

The following describes the general fields listed in the DL/I Attributes section of the Work with Process IDs area:

Field | Description
Record Length | Length of sequential unload (record).
Data Position | Position of the data area within the sequential unload that contains the information described in the copybook.
Record type position | Position of the data area within the sequential unload that contains the segment name.
Record type length | Length of the data area within the sequential unload that contains the segment name.
Type | Type of length value.
Position | Position of length value.
Length | Length of length value.
Note: The Data Length File Attributes fields provide information regarding the area, if one exists, of the sequential unload that contains the length of the data area. If values are not specified, the message “There is not enough information in the process identifier to retrieve the segment record lengths” is reported, and Data Express assumes the data area has the maximum available length (to the end of the physical layout).
Parent topic: Work with Process IDs


Available operations

The buttons at the bottom of the Work with Process IDs area are:
Refresh
Refreshes the process identifier information. If another user creates a process identifier from a different executable in the same database, you can see the changes.
Apply
Confirms the changes made to the process identifier. Before changes are applied, a series of checks on non-key data elements are performed. If the checks fail, the update is blocked.
New
Initializes fields for insertion when creating a new process identifier.
Delete
Deletes a selected process identifier. A process identifier can only be deleted if data stores have not been cataloged for it by Data Express.
Cancel
Cancels any modifications made to the selected process. Any actions confirmed with Apply cannot be restored with Cancel but must be entered and confirmed again.
OK
Saves any changes that have not been applied and closes the window.
Parent topic: Work with Process IDs


Work with Data Elements

The Work with Data Elements area lets you implement the following functions:
Associated page | Function
Select Data Elements | Find a list of data elements in the fastest and simplest way possible.
Classes - Assignment | Allow the data elements found to be associated with a class after using appropriate filters.
Estimated Classes - Confirm | Confirm (or select from suggested alternatives) the class obtained by the batch function executed for the automatic cataloging of the data elements.

To access the Work with Data Elements window, click [image] (Work with Data Elements) from the main window.

  • Window contents
Parent topic: Front End Guide


Window contents

This section describes the Work with Data Elements area as shown below:

Work with Data Elements area

Note:

The List of Data Elements grid appears a t the bottom of the window of all the pages.

  • Select Data Elements page
  • Classes - Assignment page
  • Estimated Classes - Confirm page
Parent topic: Work with Data Elements


Select Data Elements page

The Selected Data Elements page lets you filter the list of data elements by various attributes shown on the following sub-pages:

  • Data Element Attributes
  • Naming Convention
  • Data Store Attributes
  • Data Element Class
  • Available Filters area
  • Data Element Attributes page
  • Naming Convention page
  • Data Store Attributes page
  • Data Element Class page
  • Available operations
Parent topic: Window contents


Available Filters area

The Available Filters area (in the left-hand corner of the screen) is the set of principal filters used to define the environment by the following fields:

  • Machine ID
  • Company name
  • Application ID
  • Hide Data Elements
    • Assigned to Class – Hides data elements assigned to class.
    • NOT Assigned to Class – Hides data elements not assigned to class.
    • With Estimated Class – Hides data elements with estimated class.
    • Without Estimated Class – Hides data elements without estimated class.
Parent topic: Select Data Elements page


Data Element Attributes page

The following list describes the fields listed on the Data Element Attributes page:
Data Element name
Allows selection by data element name.
Data Element type
Allows selection by data element type (Alphanumeric, Numeric, Packed, Binary, Float, Edit Field, Time Stamp, Date, and Time).
Restriction: Data Express does not support LOB data element types for DB2 and ADABAS databases. Also, some Binary data element types are not handled during ADABAS FDT analysis.
Data element nature
Allows selection by data element nature (Alphabetical or Numeric).
Data Element length
Allows selection by range of data element length.
Min/Max length
Allowed length range for the class.
Min/Max integer
Allowed integer digits for the class.
Min/Max decimal
Allowed decimal digits for the class.
Register Table Fields
Allows selection based on the Register Table property.
Data Element Value
Allows selection based on the distribution of the data element contents.
Parent topic: Select Data Elements page


Naming Convention page

The following list describes the fields listed on the Naming Convention page:
Data element name contains
Allows selection of data elements whose names contain one of the strings inserted in the list.
Data element name does NOT contain
Allows non-selection of data elements whose names contain one of the strings inserted in the list.
Parent topic: Select Data Elements page


Data Store Attributes page

The following list describes the fields listed on the Data Store Attributes page:

DB2 Owner, DL/I DBD
Allows selection by file qualifier.
Data store name
Allows selection by data store name.
Data store type
Allows selection by data store type.
Data store version
Allows selection by data store version.
Parent topic: Select Data Elements page


Data Element Class page

The following list describes the fields listed on the Data Element Class page:

Super Class
Allows selection of all data elements that have an associated class of the same category as the chosen Super Class (Number, Codes, Date, Complemented Date, Description, and Other).
Assigned class
Allows selection of data elements associated with a particular class.
Estimated class
Allows selection of data elements associated with an estimated class.

By clicking Save Current Settings, views can be saved and restored at any time. Each view is given a name, which differentiates it from the others. Double-click a name to display the corresponding view. Click Delete Filter to delete a view.

Parent topic: Select Data Elements page


Available operations

The buttons on the Select Data Elements page of the Work with Data Elements area are:

Refresh
Refreshes the window.
Clear Filters
Resets all the filter conditions setup in the different tabs.
Apply Filters
Applies all filters configured to the window.
Close
Closes the window.
Parent topic: Select Data Elements page


Classes - Assignment page

The Class - Assignment page is used to assign a class to the data elements of the filtered list, supplying detailed information related to both the class and data elements. Information on the page is divided into two parts, where information linked to the classes appears on the left and to the fields on the right.

An example of the Class - Assignment page in the Work with Data Elements area is shown below:

  • All classes hierarchy
  • Class Attributes area
  • Available operations
Parent topic: Window contents


All classes hierarchy

On the left the class tree is shown. The main branches are made up of Super Classes (Number, Codes, Date, Description, Other) while the secondary ones are made up of the constituent classes.
Parent topic: Classes - Assignment page


Class Attributes area

Selecting a class updates the corresponding attribute fields (Class Attributes):

Name
Class name.
Class ID
Class code.
Description
Description of the class.
Super Class
Name of the Super Class.
Code
Identifier code for the class.
Class Samples
Examples of distribution associated with the class.

The filtered list of fields is shown at the bottom. Selecting one of these updates the corresponding attribute fields:

Selected Data Element Attributes – Data Element page

The fields listed on the Data Element page are:

Name
Name of the data element within the data store.
Type
Type of data element.
Nature
Nature of the data element (Numeric or Alphanumeric).
Displ
Position of the data element within the record layout.
Length
Length of data element in bytes.
Int
Number of integer digits.
Dec
Number of decimal digits.
Register Table Field
Indicator of whether the data element is a Register Table field.

Selected Data Element Attributes – Data Store page

The fields listed on the Data Store page are:

Name
Name of the data store.
Type
The type of data store.
Version
Version of the data store.
Number of Records
Number of records in the data store.

Selected Data Elements Attributes - General page

The fields listed on the General page are:

Machine ID
Identifier of the machine.
Company
Name of the company.
Application ID
Identifier of the application.

Selected Data Elements Attributes - Date page

The fields listed on the Date page are:

Recalculation Date
Date of last recalculation.
Parent topic: Classes - Assignment page


Available operations

The buttons listed on the Classes - Assignment page of the Work with Data Elements area are:

Refresh
Refreshes the window.
Assign Class
Assigns a class to a list of selected data elements. The steps are as follows:
  1. Select the fields involved in the association from the List of Data Elements grid.
  2. Click the Classes - Assignments tab.
  3. Select a node on the class tree.
  4. Click Assign Class.
Clear Class
Deletes the class associated with the list of selected data elements. The steps are as follows:
  1. Select the data elements involved in the disassociation from the List of Data Elements grid.
  2. Click Clear Class.
Close
Closes the window.
Parent topic: Classes - Assignment page


Estimated Classes - Confirm page

The Estimated Classes - Confirm page is used to confirm the class assignment proposals. On z/OS, this is produced by the batch analysis. Information on the page is divided into two parts, where information about proposed classes appears on the left and information about selected data elements on the right.

An example of the Estimated Classes - Confirm page in the Work with Data Elements area is shown below:

  • Class Attributes area
  • Selected Data Element Attributes area
  • Available operations
Parent topic: Window contents


Class Attributes area

On the left the list of classes proposed for the selected data element is shown. Selecting a class updates the corresponding attribute data elements:

Name
Name of the class.
Class ID
Class code.
Description
Description of the class.
Super Class
Name of the Super Class.
Code
Identifier code for the class.
Class Samples
Examples of distribution associated with the class.

The filtered list of data elements is shown at the bottom. Selecting one of these updates the corresponding attribute data elements in the Selected Data Element Attributes area.

Parent topic: Estimated Classes - Confirm page


Selected Data Element Attributes area

  • Data Element page
  • Data Store page
  • General page
  • Date page
Parent topic: Estimated Classes - Confirm page


Data Element page

The following list describes the fields listed on the Data Element page:

Name
Name of the data element within the data store.
Type
Type of data element.
Nature
Nature of the data element (for example, Numeric or Alphanumeric).
Displ
Position of the data element within the record layout.
Length
Length of data element in bytes.
Int
Number of integer digits.
Dec
Number of digits included to the right of the decimal point.
Register Table Field
Indicator of whether the data element is a Register Table field.
Parent topic: Selected Data Element Attributes area


Data Store page

The following list describes the fields listed on the Data Store page:

Name
Name of the data store.
Type
Type of data store.
Version
Version of the data store.
Number of Records
Number of records in the data store.
Parent topic: Selected Data Element Attributes area


General page

The following list describes the fields listed on the General page:

Machine ID
Identifier of the machine.
Company
Name of the company.
Application ID
Identifier of the application.
Parent topic: Selected Data Element Attributes area


Date page

The following list describes the fields listed on the Date page:

Recalculation date
Date of last recalculation.
Parent topic: Selected Data Element Attributes area


Available operations

This section describes the functions of the buttons listed on the Estimated Classes - Confirm page of the Work with Data Elements area:

Refresh
Refreshes the window.
Assign Class
Assigns a selected class to a data element. The steps are as follows:
  1. Select the field involved in the association from the List of Data Elements grid.
  2. Select the class to assign.
  3. Click Assign Class.
Confirm Class
Allows the most likely class to be assigned to a list of selected data elements. The steps are as follows:
  1. Select the fields involved in the association from the List of Data Elements grid.
  2. Click Confirm Class.
Close
Closes the window.
Parent topic: Estimated Classes - Confirm page


Work with Jobs

The Work with Jobs area is used to schedule batch jobs to run on z/OS.

Note: The Work with Jobs area is available only the Data Express for z/OS solution.

Once the product setup phase is finished and a stable state is established, the Work with Jobs area lets you work directly from a PC for both the scheduling of jobs and the consultation and maintenance of the results obtained.

To access the Work with Jobs area, click [image] (Work with Jobs) from the main window.

  • Window contents
  • Available operations
  • Job creation
  • Custom jobs
Parent topic: Front End Guide


Window contents

This section describes the Work with Jobs area as shown below:

  • List of Jobs grid
  • List of Scheduled Jobs grid
  • Window fields
Parent topic: Work with Jobs


List of Jobs grid

The List of Jobs grid displays a fixed list of possible jobs to schedule.
Parent topic: Window contents


List of Scheduled Jobs grid

The List of Scheduled Jobs grid displays a list of jobs that have been scheduled . Each scheduled job is associated with a job state as indicated by the following icons:
Icon | Description | Status flag
 | Job still to be submitted | 0
 | Job running | 1
 | Job finished | 9
 | Job not submitted | -x (negative digits)
Note: The Work with Jobs area lets you create a job without submitting it. You can submit a job later by changing its job submission status from Job Creation to Job Submission.
Parent topic: Window contents


Window fields

The following list describes the fields listed in various sections in the Work with Jobs area:
Machine ID
Identifier of the machine.
Company name
Identifier of the company.
Copy PDS
PDS of copybooks/includes to analyze.
Copybook name
Name of the copybooks/includes to analyze.
Job creation
Option for creating a new job without submitting it.
Job submission
Option for submitting a new job.
Application ID
Identifier of the application.
DB2 Owner, DL/I DBD
Qualifier of the data store to analyze.
Data store type
Type of the data store to analyze (VSAM, DB2, DL/I, SEQ, GDG, ORA, SQE, or ADA).
Data store name
Name of the data store to analyze.
Data store version
Version of GDG data store to analyze.
Record format
Record format to analyze.
Parent topic: Window contents


Available operations

This section describes the buttons at the bottom of the Work with Jobs area.

Change
Modifies a scheduled job. By selecting a scheduled job with status flag 0 (i.e. still to be submitted), all the job submission parameters are updated in the lower part of the window. It is possible to change all the parameters and then, after having introduced the changes, save them to z/OS by clicking Apply. Before updating the list, a new window is displayed requesting additional parameters.
Apply
Confirms the update/insertion data for a job to schedule.
New
Initializes all the field parameters. You can then insert the scheduling parameters of the new job. To save to z/OS, click Apply. Before updating the list, a new window is displayed requesting additional parameters.
Delete
Cancels a highlighted job. If this is being executed (flag status 1), you are prompted to confirm the delete operation.
Cancel
Cancels any changes made to the selected job. Any actions confirmed with Apply cannot be restored with Cancel but must be entered and confirmed again.
OK
Saves any changes that have not been applied and closes the window.
Parent topic: Work with Jobs


Job creation

This section describes the basic procedure for creating a new job. The subsections address the secondary parameters needed for creating specific jobs. The following guides discuss other jobs:

Life Cycle Guide
  • Life Cycle
  • Clear Life Cycle History
Data Masking Guide
  • Data Store Data Changer
  • Create Data Changer Jobs
Subset Extraction Guide
  • Test Environment Creation
  • Create Allocation Jobs
  • Create Unload Jobs
  • Create Reload Jobs
  • Create Extraction Jobs

To begin job creation:

  1. Select the job from the List of Jobs grid.
  2. Click New to initialize fields for insertion.
  3. Select Machine ID and Company name from the corresponding lists.
  4. Click Apply.
  5. Follow the guidelines in the appropriate subsection for the job you want to create.
    Note: The Reset All, Apply, and Cancel buttons are available in each Secondary Options window.
Note: If your Data Express Knowledge Base is in on MVS but the described data is on a distributed system – as an alternative to scheduling jobs using Work with Jobs – you can update the classes and the method from the referential integrity interface in interactive mode just as you would for Data Express for Distributed Systems. The interactive update is described in your Getting Started with Distributed Data Stores guide, and can be activated if:
  • The version of DB2 is 8 or later.
  • The HSURDRIA table within the Data Express Knowledge Base only contains one record and this record is set to Y. (This table must be populated by a DB2 administrator. Data Express does not directly modify the table in order to prevent unwilled use of the interactive update from referential integrity.)
  • Class Field Assignment job
  • Create Deletion Statements
  • Create Format Selector job
  • DB2 Catalog Synchronization job
  • Data Store / Copybook Relationship Analysis job
  • Data Store Data Element Sampling job
  • Export Class Assignment Information
  • Export Classes Information
  • Export Combined Data Elements Information
  • Export Copybook Information
  • Export Data Store Information
  • Export DB2 Table Information
  • Export Methods Information
  • Export Multi-format Data Stores Information
  • Load Distributed Sampling Results job
  • Import Classes Information
  • Import Classification from Data Dictionary job
  • Import Classification from Referential Integrity job
  • Import Combined Data Elements Information
  • Import Method from Referential Integrity job
  • Import Methods Information
  • Import Multi-format Data Stores Information
  • Load Copy Information from External Interface job
  • Load DB2 Table Information by Unload job
  • Load DL/I Database Information job
  • Load Data Store Information from External Interface job
  • Load SEQ/GDG Data Set Information job
  • Load VSAM Data Set Information job
Parent topic: Work with Jobs


Class Field Assignment job

Complete the values listed in the Secondary Options window for the Class Field Assignment job as shown below:

Secondary Options window

  • Sections and fields
  • Available operations
Parent topic: Job creation


Sections and fields

Complete the values in the following sections and fields as appropriate:

Select a Class
Selects a class from the list.
Super Classes
Selects a super class.
Field Types
Selects type or combination of types.
Field Length
Gives minimum and maximum length of the field.
Rules
Group of data entry functions with the masking and reduction rules to be applied to an environment.
Thresholds
Logical operations between Treshold1 and Treshold2.
Parent topic: Class Field Assignment job


Available operations

Use the Add and Delete buttons to add or delete a selected class.
Parent topic: Class Field Assignment job


Create Deletion Statements

Complete the values listed in the Secondary Options window for the Import Combined Data Elements Information Create Deletion Statements job as shown below:

Secondary Options - Create Deletion Statements

Group
The group for which you want to create deletion statements.
Method
The method for which you want to create deletion statements.
From step
The first step in a range of steps defined in conjunction with the To step parameter.
To step
The last step in a range of steps defined in conjunction with the From step parameter.
Discard Unchanged Files
From the deletion statements, discard all DB2 tables belonging to the current method that are neither reduced nor masked.
JCL output data set
The library where generated SQL statements are stored.
JCL output member
The member in which generated SQL statements are stored.
Target DB2 owner
The owner of the target environment where the statement is to run.
Important: Before executing the generated DDL:
  1. Execute a method simulation.
  2. Copy the ANENVLST table of the knowledge base into the target environment.
Parent topic: Job creation


Create Format Selector job

Complete the values listed in the Secondary Options window for the Create Format Selector job as shown below:

Create Format Selector job

Format Selector Name
The name of the format selector. It is a character field with a maximum length of 8 characters.
Position
The position of the record format. It is a 5-digit number.
Length
The length of the record format. It is a 3-digit number.

To better manage file names with format selector, KURFMT is available. Variables KURFMT-FILTYPE, KURFMT-FILNAME and KURFMT-FILVER contain characteristics of the file when using Format Selector.

The following is an example of a Format Selector program using KURFMT:

…
 COPY KURFMTW.
      *****************************************************************
       LINKAGE SECTION.
      *****************************************************************
      * INPUT
M50373*01  LK-RECORD                  PIC  X(16000).
M50373 01  LK-RECORD                  PIC  X(32760).
      * OUTPUT
       01  LK-FORMAT                  PIC  X(0010).
      *****************************************************************
       PROCEDURE DIVISION           USING LK-RECORD
                                          LK-FORMAT.
      *****************************************************************
      *****************************************************************
      *
          SET     KURFMT-OP-READ  TO TRUE.
          CALL "KURFMT" USING KURFMT-AREA.
          DISPLAY KURFMT-FILNAME
Parent topic: Job creation


DB2 Catalog Synchronization job

Complete the values listed in the Secondary Options window for the DB2 Catalog Synchronization job as shown below:

DB2 Information Interface
Type of the DB2 interface.
Parent topic: Job creation


Data Store / Copybook Relationship Analysis job

Complete the values listed in the Secondary Options window for the Data Store / Copybook Relationship Analysis job as shown below:

Data Store / Copybook Relationship Analysis

Replace Association
Indicator of whethor an existing association will be replaced.
Tolerance
This parameter allows you to specify a percentage of tolerance between the length of the data store layout and that of the copybook describing it. If you set this parameter to zero, these length values must coincide. Setting a tolerance to a value different from zero may be useful when ‘FILLER’ fields involving the use of a copybook layout longer than the data store layout are used in the copies.
Include also copy with confidence in
Choose the range.
Copy - Machine ID
Select Machine ID from the list.
Copy - Company
Select the company from the list.
Parent topic: Job creation


Data Store Data Element Sampling job

Complete the values listed in the Secondary Options window for the Data Store Data Element Sampling job as shown below:

Compressed sampling
A processing of the compressed sampling is requested.
Min/Max calculation
The minimum and maximum values of each sampled field can also be calculated.
Sampling Frequency
The frequency of the sampling execution (values from 1 to 5).
Standard sampling
Standard Sampling will be performed.

Additional field attributes can be chosen by clicking Advanced. An example of the window displayed is shown below:

Parent topic: Job creation


Export Class Assignment Information

Complete the values listed in the Secondary Options window for the Export Class Assignment Information job as shown below:

Secondary Options - Export Class Assignment Information

Sequential data store
Name of the data store containing the interface. Typically, it is a valid z/OS data store name. If it is not a valid z/OS name, it is a mnemonic, and indicates that this sequential data store should be placed in the knowledge base table HSURDINT rather than residing on disk. In this case, the mnemonic sequential data store value is placed in the HSURDINT.INTNAME column, so the contents of the data store within HSURDINT can be easily identified in further processing.
Parent topic: Job creation


Export Classes Information

Complete the values listed in the Secondary Options window for the Export Classes Information job as shown below:

Secondary Options - Export Classes Information

Sequential data store
Name of the data store containing the interface. Typically, it is a valid z/OS data store name. If it is not a valid z/OS name, it is a mnemonic, and indicates that this sequential data store should be placed in the knowledge base table HSURDINT rather than residing on disk. In this case, the mnemonic sequential data store value is placed in the HSURDINT.INTNAME column, so the contents of the data store within HSURDINT can be easily identified in further processing.
Parent topic: Job creation


Export Combined Data Elements Information

Complete the values listed in the Secondary Options window for the Export Combined Data Elements Information job as shown below:

Secondary Options - Export Combined Data Elements Information

Sequential data store
Name of the data store containing the interface. Typically, it is a valid z/OS data store name. If it is not a valid z/OS name, it is a mnemonic, and indicates that this sequential data store should be placed in the knowledge base table HSURDINT rather than residing on disk. In this case, the mnemonic sequential data store value is placed in the HSURDINT.INTNAME column, so the contents of the data store within HSURDINT can be easily identified in further processing.
Parent topic: Job creation


Export Copybook Information

Complete the values listed in the Secondary Options window for the Export Copybook Information job as shown below:

Secondary Options - Export Copybook Information

Sequential data store
Name of the data store containing the interface. Typically, it is a valid z/OS data store name. If it is not a valid z/OS name, it is a mnemonic, and indicates that this sequential data store should be placed in the knowledge base table HSURDINT rather than residing on disk. In this case, the mnemonic sequential data store value is placed in the HSURDINT.INTNAME column, so the contents of the data store within HSURDINT can be easily identified in further processing.
Parent topic: Job creation


Export Data Store Information

Complete the values listed in the Secondary Options window for the Export Data Store Information job as shown below:

Secondary Options - Export Data Store Information

Sequential data store
Name of the data store containing the interface. Typically, it is a valid z/OS data store name. If it is not a valid z/OS name, it is a mnemonic, and indicates that this sequential data store should be placed in the knowledge base table HSURDINT rather than residing on disk. In this case, the mnemonic sequential data store value is placed in the HSURDINT.INTNAME column, so the contents of the data store within HSURDINT can be easily identified in further processing.
Parent topic: Job creation


Export DB2 Table Information

Complete the values listed in the Secondary Options window for the Export DB2 Table Information job as shown below:

Secondary Options - Export DB2 Table Information

Sequential data store
Name of the data store containing the interface. Typically, it is a valid z/OS data store name. If it is not a valid z/OS name, it is a mnemonic, and indicates that this sequential data store should be placed in the knowledge base table HSURDINT rather than residing on disk. In this case, the mnemonic sequential data store value is placed in the HSURDINT.INTNAME column, so the contents of the data store within HSURDINT can be easily identified in further processing.
Parent topic: Job creation


Export Methods Information

Complete the values listed in the Secondary Options window for the Export Methods Information job as shown below:

Secondary Options - Export Methods Information

Group
The group you want to export.
Method
The method you want to export.
Sequential data store
Name of the data store containing the interface. Typically, it is a valid z/OS data store name. If it is not a valid z/OS name, it is a mnemonic, and indicates that this sequential data store should be placed in the knowledge base table HSURDINT rather than residing on disk. In this case, the mnemonic sequential data store value is placed in the HSURDINT.INTNAME column, so the contents of the data store within HSURDINT can be easily identified in further processing.
Parent topic: Job creation


Export Multi-format Data Stores Information

Complete the values listed in the Secondary Options window for the Export Multi-format Data Stores Information job as shown below:

Secondary Options - Export Multi-format Data Stores Information

Sequential data store
Name of the data store containing the interface. Typically, it is a valid z/OS data store name. If it is not a valid z/OS name, it is a mnemonic, and indicates that this sequential data store should be placed in the knowledge base table HSURDINT rather than residing on disk. In this case, the mnemonic sequential data store value is placed in the HSURDINT.INTNAME column, so the contents of the data store within HSURDINT can be easily identified in further processing.
Parent topic: Job creation


Load Distributed Sampling Results job

Complete the values listed in the Secondary Options window for the Load Distributed Sampling Results job as shown below:

This job can be used when knowledge base is DB2 in order to load sampling result into knowledge base via a batch job running on MVS. In this way the sampling runs on distributed environment, it generates a log file containing sampling result, this log file should be moved on MVS (in a FB sequential file with record length equal to 80) and it is processed by the Load Distributed Sampling Result job. The parameter for this job is the sampling data set name (the sequential file containing sampling result).

Parent topic: Job creation


Import Classes Information

Complete the values listed in the Secondary Options window for the Import Classes Information job as shown below:

Secondary Options - Import Classes Information

Sequential data store
Name of the data store containing the interface. Typically, it is a valid z/OS data store name. If it is not a valid z/OS name, it is a mnemonic, and indicates that this sequential data store should be placed in the knowledge base table HSURDINT rather than residing on disk. In this case, the mnemonic sequential data store value is placed in the HSURDINT.INTNAME column, so the contents of the data store within HSURDINT can be easily identified in further processing.
Parent topic: Job creation


Import Classification from Data Dictionary job

Complete the values listed in the Secondary Options window for the Import Classification from Data Dictionary job as shown below:

Dictionary Interface
The dictionary interface name.
Parent topic: Job creation


Import Classification from Referential Integrity job

Complete the values listed in the Secondary Options window for the Import Classification from Referential Integrity job as shown below:

Secondary Options - Import Classification from Referential Integrity

Referential Integrity Interface
Referential Integrity Interface name.
Parent topic: Job creation


Import Combined Data Elements Information

Complete the values listed in the Secondary Options window for the Import Combined Data Elements Information job as shown below:

Secondary Options - Import Combined Data Elements Information

Sequential data store
Name of the data store containing the interface. Typically, it is a valid z/OS data store name. If it is not a valid z/OS name, it is a mnemonic, and indicates that this sequential data store should be placed in the knowledge base table HSURDINT rather than residing on disk. In this case, the mnemonic sequential data store value is placed in the HSURDINT.INTNAME column, so the contents of the data store within HSURDINT can be easily identified in further processing.
Parent topic: Job creation


Import Method from Referential Integrity job

Complete the values listed in the Secondary Options window for the Import Method from Referential Integrity job as shown below:

Secondary Options - Import Method from Referential Integrity

Referential integrity interface
Referential Integrity Interface name.
Method
Name of a particular method or all methods.
Method text
Method description.
Class
Name of the class.
Update method
Check this to update an existing method, adding a new relationship between data stores.
Parent topic: Job creation


Import Methods Information

Complete the values listed in the Secondary Options window for the Import Methods Information job as shown below:

Secondary Options - Import Methods Information

Group
The group you want to import.
Method
The method you want to import.
Sequential data store
Name of the data store containing the interface. Typically, it is a valid z/OS data store name. If it is not a valid z/OS name, it is a mnemonic, and indicates that this sequential data store should be placed in the knowledge base table HSURDINT rather than residing on disk. In this case, the mnemonic sequential data store value is placed in the HSURDINT.INTNAME column, so the contents of the data store within HSURDINT can be easily identified in further processing.
Parent topic: Job creation


Import Multi-format Data Stores Information

Complete the values listed in the Secondary Options window for the Import Multi-format Data Stores Information job as shown below:

Secondary Options - Import Multi-format Data Stores Information

Sequential data store
Name of the data store containing the interface. Typically, it is a valid z/OS data store name. If it is not a valid z/OS name, it is a mnemonic, and indicates that this sequential data store should be placed in the knowledge base table HSURDINT rather than residing on disk. In this case, the mnemonic sequential data store value is placed in the HSURDINT.INTNAME column, so the contents of the data store within HSURDINT can be easily identified in further processing.
Parent topic: Job creation


Load Copy Information from External Interface job

Complete the values listed in the Secondary Options window for the Load Copy Information from External Interface job as shown below:

Secondary Options - Load Copy Information from External Interface

Sequential file
Name of the sequential file on z/OS.
Parent topic: Job creation


Load DB2 Table Information by Unload job

This job can be used before creating the extraction method. Be sure to specify a sequential unload of the files and insert the process identifier for reading the file.

Complete the values listed in the Secondary Options window for the Load DB2 Table Information by Unload job as shown below:

Insert the fields related to the sequential unload:

SYSPUNCH or DCLGEN name
The name of the SYSPUNCH or DCLGEN file.
Unload data set name
The name of the unload data set.
Process identifier
Process identifier used for the management of DB2 files with sequential access.
Application identifier
Application identifier (if applicable).

For the z/OS solution, the only DB2 utility for the UNLOAD of table supported by Data Express is DSNTIAUL or a direct JCL Unload with some limitations.

The following is a sample of the JCL:

“//DSURUNL2  JOB (00001),DMP,MSGCLASS=A,CLASS=1,NOTIFY=&SYSUID
//JOBLIB   DD DISP=SHR,DSN=DSN810.SDSNLOAD
//* ---------------------------------------------------------------- *
//UNLOAD3  EXEC PGM=IKJEFT01,DYNAMNBR=20
//STEPLIB  DD DISP=SHR,DSN=DSN810.SDSNLOAD
//SYSTSPRT DD  SYSOUT=*
//SYSPRINT DD SYSOUT=*
//SYSUDUMP DD SYSOUT=*
//SYSTSIN  DD  *
 DSN SYSTEM(HADB)
 RUN  PROGRAM(DSNTIAUL) PLAN(DSNTIB81) -
       LIB('DB2HAL.RUNLIB.LOAD')
//SYSREC00  DD DSN=URADAR.RUMYANA.SYSREC.D44444,
//             DISP=(NEW,CATLG,CATLG),
//             UNIT=SYSDA,SPACE=(TRK,(30,30))
//SYSPUNCH  DD DSN=URADAR.RUMYANA.SYSPUN(D44444),
//             DISP=OLD
//SYSPRINT DD SYSOUT=*
//SYSIN    DD *
             "D444444444444444444444444444444444444444444444444444444444
4444444444444444444444444444444444444444444444444444444444444444444444".
"D4444444444444444444444444444444444444444444444444444444444444444444444
444444444444444444444444444444444444444444444444444444444"
“

The following is a sample SYSPUNCH generated by the JCL described above:

“
  LOAD DATA LOG NO INDDN SYSREC00 INTO TABLE
      "D4444444444444444444444444444444444444444444444444444444444444444
444444444444444444444444444444444444444444444444444444444444444"."D44444
444444444444444444444444444444444444444444444444444444444444444444444444
44444444444444444444444444444444444444444444444444"
   (
   CARD_TYPE                              POSITION(       1         )
   CHAR(                      1) ,
   CARD_NUM                               POSITION(       2         )
   CHAR(                     16) ,
   ACCOUNT_OPER                           POSITION(      18:      30)
   DECIMAL                       ,
   DATE_OPRT                              POSITION(      31         )
   CHAR(                      8)
   )
“

The following is a sample of the UNLOAD's JCL:

“//DSURUNL8  JOB (00001),DMP,MSGCLASS=A,CLASS=1,NOTIFY=&SYSUID
//JOBLIB   DD DISP=SHR,DSN=DSN810.SDSNLOAD
//* ---------------------------------------------------------------- *
//UNLOAD3  EXEC DSNUPROC,SYSTEM=HADB,UID='UNLOA1',
//  UTPROC=''
//STEPLIB  DD DISP=SHR,DSN=DSN810.SDSNLOAD
//SYSREC    DD DSN=URADAR.STEFANO.SYSREC.PIPPO,
//             DISP=(NEW,CATLG,CATLG),
//             UNIT=SYSDA,SPACE=(TRK,(30,30))
//SYSPUNCH  DD DSN=URADAR.STEFANO.SYSPUNCH.PIPPO,
//             DISP=(NEW,CATLG,CATLG),
//             UNIT=SYSDA,SPACE=(TRK,(1,1))
//SYSIN     DD *
    UNLOAD FROM TABLE
 NRT40NAME_TESTXM.TEST_LENGTH37CHARACT
 HEADER NONE
“

The previous JCl is supported only for those tables having owner name and or table name not longer than 40 character.

The following SYSPUNCH sample is the only format generated by the JCL described above which is supported by Data Express:

“
LOAD DATA INDDN SYSREC   LOG NO  RESUME YES
 EBCDIC  CCSID(00037,00000,00000)
 INTO TABLE
 "NRT40NAME_TESTXM".
 "TEST_LENGTH37CHARACT"
 ( "CODCUS"
  POSITION(  00001:00010) CHAR(00010)
 , "NAME"
  POSITION(  00011:00040) CHAR(00030)
 , "SURNAME"
  POSITION(  00041:00070) CHAR(00030)
 )
 “

The following SYSPUNCH sample is the BMC UNLOAD format which is supported by Data Express:

LOAD DATA LOG NO INDDN SYSREC00 INTO TABLE
    OWURSVIL.BMCTAB99
 (
 CODCUS99                               POSITION(        1:        5)
 DECIMAL
      NULLIF BMC_NULL1=X'6F',
      BMC_NULL1 POSITION(6) CHAR (1),
 NAME99                                 POSITION(        7:       26)
 CHAR (20)
      NULLIF BMC_NULL2=X'6F',
      BMC_NULL2 POSITION(27) CHAR (1),
 SURNAME99                              POSITION(       28:       47)
 CHAR (20)
      NULLIF BMC_NULL3=X'6F',
      BMC_NULL3 POSITION(48) CHAR (1),
 ADDRESS99                              POSITION(       49:       88)
 CHAR (20)
      NULLIF BMC_NULL4=X'6F',
      BMC_NULL4 POSITION(89) CHAR (1),

The following SYSPUNCH sample is a format generated by the JCL described above which is not supported by Data Express:

“
LOAD DATA INDDN SYSREC   LOG NO  RESUME YES
 EBCDIC  CCSID(00037,00000,00000)
 SORTKEYS                  72
 INTO TABLE
 "D444444444444444444444444444444444444444" ||
 "4444444444444444444444444444444444444444" ||
 "4444444444444444444444444444444444444444" ||
 "44444444".
 "D444444444444444444444444444444444444444" ||
 "4444444444444444444444444444444444444444" ||
 "4444444444444444444444444444444444444444" ||
 "44444444"
 ( "CARD_TYPE"
  POSITION(  00001:00001) CHAR(00001)
 , "CARD_NUM"
  POSITION(  00002:00017) CHAR(00016)
 , "ACCOUNT_OPER"
  POSITION(  00018:00030) DECIMAL
 , "DATE_OPRT"
  POSITION(  00031:00038) CHAR(00008)
 )
 “
Parent topic: Job creation


Load DL/I Database Information job

This job can be used before creating the extraction method. Be sure to specify a sequential unload of the files and insert the process identifier for reading the file.

Complete the values listed in the Secondary Options window for the Load DL/I Database Information job as shown below:

Secondary Options - Load DL/I Database Information

DBD name
The name of the DL/I database.
Unload data set name
The name of the unload data set.
Process identifier
Process identifier used for the management of DL/I files with sequential access.
Application identifier
Application identifier (if applicable).
Select Option
Select this option if partial unload of the multi format segments will be performed. In this case, define the Segment Name and Format Selector of the segments you want to be unloaded.
Parent topic: Job creation


Load Data Store Information from External Interface job

Complete the values listed in the Secondary Options window for the Load Data Store Information from External Interface job as shown below:

Secondary Options - Load Data Store Information from External Interface

Sequential file
Name of the sequential file on z/OS.
Note: Before starting the job, all copybooks described in the sequential file must be already loaded in the product.
Parent topic: Job creation


Load SEQ/GDG Data Set Information job

Complete the values listed in the Secondary Options window for the Load SEQ/GDG Data Set Information job as shown below:

Data set name
Name of the data set on z/OS.
Short DSN
Short name of the data set.
Unload data set name
Name of the unload data set. This is for the z/OS environment only. It can be different from the original name loaded into the Knowledge Base. To do this, the process ID associated with the elaboration should be created with characteristics for an Unload File Access. For example, GENUN.
Process identifier
The routines needed by the product to read, analyze and interpret the data of the processed files.
Format selector
Exit routine for recognizing formats.
Application identifier
Application identifier (if applicable).
Parent topic: Job creation


Load VSAM Data Set Information job

Complete the values listed in the Secondary Options window for the Load VSAM Data Set Information job as shown below:

Secondary Options - Load VSAM data set information

VSAM data set name
Name of the VSAM data set on z/OS.
Short DSN
Short name of the data set.
Unload data set name
Name of the unload data set.
Process identifier
The routines needed by the product to read, analyze and interpret the data of the processed files.
Format selector
Exit routine for recognizing formats.
Application identifier
Application identifier (if applicable).
Parent topic: Job creation


Custom jobs

Using Work with Jobs, you can create custom jobs that can be submitted by the z/OS engine just as the default preloaded jobs are submitted. To so this, Data Express provides a wizard into which you put a job name, the name of the MVS skeleton used to execute the JCL, a job description, and job parameters.

You can choose the job parameters from a list that enables you to select the scope of various Data Express operations. The list contains the following parameters:

  • Machine ID
  • Company
  • Copybook PDS
  • Copybook name
  • Application ID
  • DB owner or DL/I DBD
  • Data store type
  • Data store name
  • Data store version
  • Record format
  • Method
  • Group
  • Code

In addition, you can create custom parameters. You must supply the following information for each custom parameter you create:

  • Name
  • Description
  • Length
  • Type (alphanumeric or numeric)
Parent topic: Work with Jobs


New Custom Query Wizard

The New Custom Query Wizard guides you through creating custom views for the data stores in the Knowledge Base.

  • Starting the New Custom Query wizard
Parent topic: Front End Guide


Starting the New Custom Query wizard

  1. To start the New Custom Query Wizard, click [image].
    Note: To return to the Set Name&Bitmap screen and discard any changes, click Restart. To exit the wizard and discard any changes, click Abort.
  2. Specify the query name.
  3. Select the image you want to use by clicking the Button Bitmap image.
  4. Click Next.
  5. Select the fields to be displayed in the custom query.
    Note: Click Finish if you do not want to set a filter. The function proceeds with creating the view.
  6. Click Next.
  7. Filter results for the custom query by selecting information for the following fields:
    • Machine ID
    • Company name
    • Application ID
    • DB2 Owner DL/1 DBD
    • File type
    Note: Click Finish if you do not want to set any other filter. The function proceeds with creating the view.
  8. Click Next.
  9. Select the desired filters and click Add.
    Note: Click Finish if you do not want to sort the query. The function proceeds with creating the view.
  10. Click Next.
  11. Set the order by which you want to display data.
  12. Choose one or more fields from the List of available data elements list and the Sort Mode section.
    Note: Click Finish if you do not want to set the query as default. The function proceeds with creating the view.
  13. Click Next.
  14. If desired, check Set this query as default view on startup to specify the query you created is used as the default.
  15. Click Finish.
Parent topic: New Custom Query Wizard


Options

Data Express can be customized to meet the requirements for different users.

This section describes the available options in the Data Builder and Data Subset Extraction modules. For available options in the Data Masking module, see the Data Masking Guide.

  • Window contents
  • Available operations
Parent topic: Front End Guide


Window contents

Options can be edited in the Options window. To display the window, click Tools > Options.

  • General
  • Filter settings
  • Confirmation messages
  • Confirmation routine name
  • Server information
Parent topic: Options


General

The General page has the following fields:

Help options |
Help language | The language used for online help. The Help language list contains the languages available on your PC.
Security options |
Save user and password | An indicator of whether or not you want to save your user ID and password information for the current database connection.
Parent topic: Window contents


Filter settings

The Filter Settings page allows the scope of the filters configured in the Work with Data Stores area to be selected.

The Filter Settings page has the following fields:

Filter Scope Options |
Filter on current TabSheet | This filter is valid for the current sheet, so that the filter is applied if the tree node is changed, but is not applied if the sheet is changed.
Filter on all TabSheets | This filter is valid for the nodes of the tree, so that the filter is not applied when the tree node is changed.
Data Store Behavior Options |
Show filter before loading the data | Allows the filter to be shown before loading the data.
Clear All Existing Filters | Deletes all existing filters.
Parent topic: Window contents


Confirmation messages

The Confirmation Messages page lets you set specific work modes. The Confirmation Messages page has the following fields:

Assign the example data element to all companies – (Yes, No, Ask).
Clean Class/es assigned to the selected data elements – (Yes, Ask).

Parent topic: Window contents


Confirmation routine name

The Confirmation Routine Names page lets you set specific work modes. The Confirmation Routine Names page has the following fields:

Check the existence of masking/subsetting routine – (Yes, No, Ask).

Value Yes means that in any place you can specify a routine, you can only choose a routine inserted using Work with Routines.

Parent topic: Window contents


Server information

The Server Information page allows the connection options for Data Express kbde-Server to be displayed and modified.

Note: The Server Information page is only displayed in Client/Server mode use of the Data Builder module.

The Server Information page has the following fields:

Server Information |
IP Address | The IP address of the NT server (required for Socket and DCom connections).
Port Number | The port number assigned to the Borland Socket Server validity when the tree node is changed (required for Socket connections).
URL | The URL to identify the httpsrvr.DLL file installed on the server machine (required for HTTP connections).
Connection options | Type of connection (Socket, DCom, HTTP).
Parent topic: Window contents


Available operations

The buttons at the bottom of the Options window are:

OK
Saves any changes that have not been applied and closes the window.
Cancel
Cancel any changes applied. Any actions confirmed with Apply cannot be restored with Cancel but must be entered and confirmed again.
Apply
Confirms the changes.
Parent topic: Options


File Properties

The Properties - File window is a tabbed view that displays all attributes for a selected data store.

To access the Properties - File window, click Data Store Properties Button (Data Store Properties) from the Work with Data Stores area, or right-click the appropriate table from the List of Data Stores and click Data Store Properties.

  • Window contents
  • Available operations
Parent topic: Front End Guide


Window contents

This section describes each tab of the Properties - File window.
  • General attributes page
  • System page
  • Object attributes page
  • Dates page
  • Associated Copybook page
Parent topic: File Properties


General attributes page

The fields on the General Attributes page are:
Data store description
Description of the data store.
Application ID
Identifier (if it exists) of the application area that the data store belongs to.
Process ID
Process Identifier.
Format selector
Exit routine for data-store format recognition.
Record length
Length (in bytes) of the record.
Data store name
Name of the data store.
Data store type
Type of data store, for example, DB2, VSAM, DL/I, GDG, SEQ, ODBC, or ORA.
Data store ver
Version of the data store.
Sampling frequency
Frequency (for example, daily or monthly) of sampling of the table fields for which the distribution of values is desired. (Only applicable for Data Express for z/OS.)
Register Tables
Indicator of whether the data store is a register table.
Parent topic: Window contents


System page

The fields on the System page are:

Machine ID
Identifier of the machine.
Company name
Name of the company.
Parent topic: Window contents


Object attributes page

The fields on the Object Attributes page are:
Data store type
Type of table, for example, DB2, VSAM, DL/I, GDG, SEQ, ODBC, ORA.
Data store name
Name of the data store.
Data store description
Description of the data store.
Application ID
Identifier (if it exists) of the application area that the data store belongs to.
Record length
Length (in bytes) of the record.
Record format
Name of the record format.
Format selector
Exit routine for table format recognition.
Number of records
Number of records contained in the data store.
Analyzed records
Number of analyzed records.
Database name
Name of the database.
Tablespace name
Name of the tablespace.
Object ID
Object identifier.
Parent topic: Window contents


Dates page

The fields on the Dates page are:

Date of analysis
Date of the analysis carried out on the table.
Date of last calculation
Date of the last recalculation carried out on the table.
Date of last sample
Date of the last calculation of the distribution of values of the table fields involved in the process.
Parent topic: Window contents


Associated Copybook page

The Associated Copybook page is only available in the Data Express for z/OS solution.

The fields on the Associated Copybook page are:

Copy PDS
Partitioned data set to which the copybook assigned to the data store belongs.
Copybook name
Name of the copybook assigned to the data store.
I/O area name
Name of the area present on the copybook which describes the layout of the data store.
I/O Area Progressive
Progressive number of the area within the copybook associated with the data store.
Information provider
Identifies who carried out the association of the copybook with the data store. Manual indicates the user did this manually. Automatic indicates the assignment was done automatically by the analysis carried out Data Express.
Note: The Associated Copybook page is not visible for DB2 tables.
Parent topic: Window contents


Available operations

The buttons at the bottom of the Properties – File window are:

Change
Enables control and goes into edit mode.
Note: This button is only available on the General Attributes page.
OK
Saves any changes that have not been applied and closes the window.
Cancel
Cancels any changes applied to the data store properties and closes the window. Any actions confirmed with Apply cannot be restored with Cancel but must be entered and confirmed again.
Parent topic: File Properties


Filter by Data Store

The Filter-Data Stores window enables you to apply a filter definition to the data store list so you can make a partial selection of information and carry out a one-off analysis.

Various types of selection can be applied and are applicable to different requirements. For example:

  • A very large application might be divided by Application ID values so each manager could carry out an analysis on his area of responsibility.
  • Anyone who uses a prefix for distinguishing data store belonging to an area could use a partial selection on the data store name.
  • Anyone who uses a different qualifier (for example, Owner) for each area of responsibility of data stores could carry out a selection by library.

To access this function, click Filter-Data Stores Button (Filter-Data Stores) from the Work with Data Stores area.

  • Window contents
  • Available operations
Parent topic: Front End Guide


Window contents

This section describes the fields listed on each page of the Filter-Data Stores window.
  • System page
  • Object page
  • Record page
  • Dates page
  • Associated Copybook page
  • Scheduling page
Parent topic: Filter by Data Store


System page

The fields on the System page are:

Machine ID
Selects by machine ID.
Company
Selects by company name.
Parent topic: Window contents


Object page

The fields on the Object page are:
Application ID
Selects data stores based on the application area they belong to.
DB2 owner, DL/I DBD
Selects by (DB2) owner or by database description (DL/I).For other types of data stores, the generic term *ALL can be used.
Data store type
Selects by data store type distinguishing between DB2, VSAM, DL/I, Sequential (SEQ), GDG, ODBC, or ORA.
Data store name
Selects by data store name.
Data store version
Selects by data store version.
Analyzed
Selects by state of analysis of the data store. The possible values are: (Yes, No, and *ALL).
Parent topic: Window contents


Record page

The fields on the Record page are:
Record format
Selects by record format.
Record length
Selects data stores with record layout length within a range of values. If only one of the two limits of the interval is specified all of the values greater than the start value (if the end value is not specified) or less than the end value (if the start value is not specified) are extracted.
Number of records
Selects data stores with the number of records within a range of values. If only one of the two limits of the interval is specified all of the values greater than the start value (if the end value is not specified) or less than the end value (if the start value is not specified) are extracted.
Records analyzed
Selects data stores with the number of records, which have been analyzed within a range of values. If only one of the two limits of the interval is specified, all of the values greater than the start value (if the end value is not specified) or less than the end value (if the start value is not specified) are extracted.
Parent topic: Window contents


Dates page

The fields on the Dates page are:
Date of analysis
Selects data stores with a given analysis date.
Date of last calculation
Selects data stores with a given date of the last recalculation.
Date of last sample
Selects data stores with a given date of the last calculation of the distribution of values.
Parent topic: Window contents


Associated Copybook page

The fields on the Associated Copybook page are:
Copybook PDS
Selects by partitioned data set the copybooks assigned to the data stores.
Copybook name
Selects by the name of copybooks assigned to the data stores.
Parent topic: Window contents


Scheduling page

The Scheduling page only applies to Data Express for z/OS.

The field on the Scheduling page is:

Sampling frequency
Selects by frequency of sampling (for example, daily or monthly) of the data store data elements.
Parent topic: Window contents


Available operations

The buttons at the bottom of the Filter-Data Stores window are:

Clear All
Clears all modifications made to the selected filter.
Settings
Sets any modifications made to the selected filter.
OK
Saves any changes made to the selected filter and closes the window.
Cancel
Cancels any changes made to the selected filter and closes the window. Any actions confirmed with Apply cannot be restored with Cancel but must be entered and confirmed again.
Parent topic: Filter by Data Store


Compare Layouts - Copybook

The Compare Layouts– Copybook window lets you check that the calculated association is correct, by checking the byte-by-byte congruence of the data element layout with an associated copybook.

Note: The Compare Layouts– Copybook window is not applicable for distributed data stores.

Data Express uses an association algorithm to determine the list of compatible copybooks (that describe its trace) for each data store. The association algorithm is based on:

  • Compatibility of length between data store and copybook.
  • Byte-by-byte congruence between the data type of the data store and copybook traces.

The information for copybook bytes is obtained from the analysis of the source of the same copybook. The information for file bytes is obtained during the file analysis on the basis of the congruence of the values found within the records, their positions being equal.

Each byte, depending on its type, has a specific hexadecimal content. The evaluation of the congruence between the copybook and the file bytes is based on an algorithm comparing the copybook byte with the hexadecimal content of the corresponding file byte.

To access this function, click [image] (Compare Layouts – Copybook) from the Work with Data Store area.

  • Window contents
  • Available operations
Parent topic: Front End Guide


Window contents

This section describes the fields listed in the Compare Layouts – Copybook window.

The upper part of the window shows two frames containing the file and the selected copybook attributes. Selecting the assigned copybook node on the left-hand side displays a list of compatible copybooks.

Field | Description
Data Store
Machine ID | Identifier of machine that the data store belongs to.
Company | Identifier of company that the data store belongs to.
Name | Name of the data store.
Record format | Record format name.
Record length | Length of the data store record.
Selected Copybook
Machine ID | Identifier of machine that the data store belongs to.
Company | Identifier of company that the data store belongs to.
PDFS | Partitioned data set that the selected copybook belongs to.
Name | Name of the selected copybook.
Copy Length | Length of the selected copybook.
I/O Area Name | Name of the I/O area identified within the copybook.
Prog | I/O area progressive number.
Byte-by-Byte Congruence
Byte Position | Position of the seventh byte in the data store layout.
File Byte | Data type for the seventh byte in the data store layout. Available values: Alphanumeric, Binary, Filler, Low-value, Packed Central, Packed Final Unsigned, Packed Final Signed, Space, Zoned Unsigned, Zoned Signed, and Bit.
Confidence | Hypothesis confidence for the seventh byte in the data store layout.
Copybook byte The seventh byte type of the selected copybook. Available values: Alphanumeric, Binary, Filler, Low-value, Packed Central, Packed Final Unsigned, Packed Final Signed, Space, Zoned Unsigned, Zoned Signed, and Bit.
Green disk icon
Full compatibility, the copybook byte is the same as the file byte.
Half green, half yellow disk icon
Partial compatibility with high reliability.
Yellow disk icon
Partial compatibility with low reliability. The file bytes always set to spaces and/or the copybook bytes included in filler areas also belong to this category.
Red disk icon
Total incongruence between the copybook byte and the file byte.
Level of compatibility indicators
Copy Bytes Matching | Percentage of bytes compatible between data store and copybook with respect to the length of the data store layout.
Confidence | Estimate of the level of similarity between the proposed file byte value and the corresponding file byte content considering the file analysis only.
Compatible Copybooks | Displays the list of all the copies with the same level of compatibility with the data store.
All Copies (Filtered) | Displays all the copybooks belonging to the environment selected on the basis of the specified associated filters.
Filter on All Copybooks - General Tab
Machine ID | Allows selection by machine identifier.
Company
Filter on All Copybooks - Copybook Attributes Tab
Name | Allows selection by company name.
Tolerance % | If a value different from 0 (zero) is set, it allows you to select copies from the environment whose length differs from that of the file layout by the set percentage value. By default, the parameter is set to 0. This implies that copies whose length coincides with that of the file are displayed. Value 100 allows you to select all the copybooks belonging to the Machine ID and to the company selected on the General page.
Parent topic: Compare Layouts - Copybook


Available operations

The buttons listed in the Compare Layouts – Copybook window are described below:
Button | Description
Data Store |
Compare | Displays a list showing the byte-per-byte compatibility between data store and selected copybook and the Copy Bytes Matching and Confidence percentages.
Assign | Assigns the copybook to the data store.
Delete | Deletes the copybook-data store association.
Graph Button (Graph) | Opens a window including a pie chart of the distribution of the byte compatibility between data store and selected copybook.
Filter |
Clear Filters | Returns all the conditions included in the window to their initial value.
Apply Filters | Applies the selected filters.
Close | Closes the window.
Parent topic: Compare Layouts - Copybook


Sample Graph - Data Sampling

The Data Sampling function can be used to display a pie chart of the distribution of values of a data store data element, highlighting the values that occur most often within the data element.

To access the Sample Graph window from the Data Sampling function:

  1. Click the [image] tab or [image] (View Data Stores by Sampling) on the toolbar.
  2. Right-click the appropriate table and click Open Selected Data Store.
  3. Right-click the appropriate table in the List of Data Elements and click Show In Sampling.
  4. Select the appropriate field in the List of Sample Analysis of the Selected Data Store Field, and either click [image] (Sample Graph of the Selected Data Element) or right-click to display the context menu.
  5. Click Sample Graph of the Selected Data Element.
  • Window contents
Parent topic: Front End Guide


Window contents

This section describes the contents of the Sample Graph window.

The pie chart shows the values that occur most often.

On the right-hand side, the legend shows how many times a certain value appears and the color of its corresponding segment in the graph. The width of each segment is proportional to the number of records with that value.

The fields and buttons in the Sample Graph window are:

Number of Different Values
Number of different values estimated for this field.
Show the x Values with Major Occurrence
Number of largest occurring different values. By default, this value is 5. All the other values are included in the segment labeled as OTHER. This value can be increased or decreased.
Apply
Applies the changed value in the field Values with Major Occurrence and shows the graph with the changed values immediately.
OK
Closes the window.
Parent topic: Sample Graph - Data Sampling


Show Synthetic Data Element Contents

The function of the Show Synthetic Data Element Contents window is to obtain a concise graphical representation of data element content. The representation of the content depends on the nature of the data element.

  • The values of the numeric data elements are divided into homogeneous groups that are differentiated by the value of their first digit and by the number of digits they contain. This description preserves the general representation of the distribution of the values throughout the data element.
  • The values of the alphanumeric data elements are divided according to various criteria:
    • First character
    • Value length
    • Presence of numeric characters only
    • Presence of alphabetical characters only

The number of values belonging to each class is stored for all of these criteria. All this information allows you to preserve many of the data elements content properties.

To access the Show Synthetic Data Element Contents window, from the main window, select the appropriate data store and either click Show Synthetic Data Elements Button (Show Synthetic Data Elements). Or, right-click to display the context menu, then click Show Synthetic Data Element Contents.

  • Window contents
Parent topic: Front End Guide


Window contents

This section describes the fields listed in the Show Synthetic Data Element Contents window.

The tree of the data store data elements for which the sampling has been carried out is displayed on the left-hand side of the window. If you select one of these data elements, its concise content in the form of a table and graph is highlighted on the right-hand side of the window.

Control | Description
Field: |
Machine ID | Identifier of machine that the data store belongs to.
Company | Identifier of company that the data store belongs to.
Name | Name of the data store.
Record format | Name of the record format.
Data store type | Type of data store.
Number of records | Number of records in the data store.
Recalculation date | Reference sampling date.
Button: |
Apply | Applies the filter on a recalculation date.
Close | Closes the window.
Sample or Graph | Shows the synthetic data element content in the form of a value list and of a graph at the bottom of the window.
Sort or Normal | Orders the sampling density values in decreasing order (valid for numeric data elements only).
Filter | Opens a window where you can apply filters to the data element selection.
Parent topic: Show Synthetic Data Element Contents


Work with Copybooks

The Work with Copybooks area provides information on the COBOL copies and PL/I includes loaded in the Data Express Knowledge Base.

Note: The languages handled by Data Express are COBOL, PL/I, and DCLGEN. Any sources written in different languages are loaded as unhandled languages. If you want to avoid loading unrecognized sources, choose the relevant option.

To access the Work with Copybooks area, click Work with Copybooks Button (Work with Copybooks) on the toolbar.

  • Window contents
Parent topic: Front End Guide


Window contents

This section describes the columns listed in the Work with Copybooks area and Work with a Selected Copybook window.

Work with Copybooks

Copybook Name
Name of the COBOL copybook or of the PL/I include.
Copybook PDS
Name of the partitioned data set that contains the COBOL copybook or PL/I include file.
Machine ID
Identifier of the machine that the copybook belongs to.
Company
Identifier of the company that the copybook belongs to.
I/O area name
Name of the I/O area identified within the copybook.
IO Area Prog
I/O area progressive number determined by incongruent redefinitions of the area.
Copy Text
The copybook text.
Analysis Flag
Indicator of whether the copybook has been analyzed.
Bad Copy Flag
Indicator of whether there is additional information on the copybooks.
Type of Anomaly
MIX (1)
Statements have been identified within the copybook.
N01 (2)
The 01 level of the I/O area has not been identified.
MUL (3)
More than an I/O area has been identified within the copybook.
NFR (4)
No I/O areas have been identified within the copybook.
ERR (5)
An error has been identified during the analysis.
Language
Copybook language:
CBL
COBOL copy
PLI
PL/I include
DGC
COBOL dclgen
DGP
PL/I dclgen
DDL
DDL
N/A
Non-handled language
Record Length
Length of the copybook records.
Number of Fields
Number of fields in the copybook/include file.
Number of Packed Fields
Number of packed fields in the copybook/include file.
Number of Binary Fields
Number of binary fields in the copybook/include file.
Associated Copy
Indicator of whether the I/O area has been associated with a data store.

Work with Selected Copybook

Field name
Name of the field.
Displacement
Field offset at the beginning of the I/O area.
Field Level
Definition level of the field within the COBOL copybook or PL/I include.
Type
Data type included in the field:
(B)
Binary numeric type.
(F)
Floating point numeric type.
(N)
Zoned numeric type.
(P)
Packed numeric type.
(X)
Alphanumeric type.
(Z)
Edited type.
(A)
Pointer type.
Length
Length of the field in bytes.
Integer
Number of integer digits within the numeric field.
Decimal
Number of decimal digits within the numeric field.
Sign
Indicator of whether the numeric field is signed (Yes or No).
OCCURS Level
Indicator of whether the field is an array.
Group
Indicator of whether the field belongs to a group (Yes or No).

Buttons

Refresh
Refresh Button Refreshes the screen.
Open selected Copy
Opens another menu with detailed information about the selected copybook/include file and fields contained in it.
Delete Selected Copybook
Deletes the selected copybook.
Parent topic: Work with Copybooks


Distributed Loader

The Distributed Loader is a utility that lets you load and map information from source data stores to your Knowledge Base as part of the Data Inventory process. The Knowledge Base can exist either on Windows or z/OS.

Note:
  • For data stores that you will access using the Oracle Extension, you must configure a BDE alias for your data store.
  • For data stores that you will access using the ODBC Extension, only an ODBC DSN is required.

For more information, see the Getting Started with Distributed Data Stores guide.

You can use the Load from Database functionality if you have a connection configured to access your data store from your Windows machine. Otherwise, you must use the Load from File functionality.

To access the Distributed Loader window, click Distributed Loader Button from the main window.

  • Load from database
  • Load from file
Parent topic: Front End Guide


Load from database

Begin by selecting the Load from Database tab. In Data Express for Distributed Systems, you will see an import checkbox for referential integrity labeled: Load Referential Integrity. This provides the ability to load your referential integrity information into the Windows Knowledge Base, in addition to standard schema information about your tables. You are now ready to import your referential integrity classes. See the Getting Started with Distributed Data Stores guide for more information.

In Data Express for z/OS, an export checkbox labeled Export Referential Integrity is provided instead. In this environment, a file that contains referential information is created for you. Once this file is transferred to the mainframe, you can load and import your referential integrity classes. See the Getting Started with Distributed Data Stores guide for more information.

Standard schema information about your distributed data store’s tables is automatically loaded in the z/OS knowledge base. You can logically reduce the amount of information you wish to load by specifying the name of a schema in the Schema field. Otherwise, you leave this field blank and the Distributed Loader retrieves and loads all the information about all the schemas in the data store.

Parent topic: Distributed Loader


Load from file

Begin by selecting the Load from File tab. In Data Express for Distributed Systems, two files can be used as input to the load process. The first file contains information about your tables, columns, and indexes. The second optional file contains information about your data store’s referential integrity.

In Data Express for z/OS, only the file that contains information about your tables, columns, and indexes can be used. For a z/OS Knowledge Base, your referential integrity information gets loaded into the Knowledge Base while executing the Import Classification from Referential Integrity job.

The files used as input by the Distributed Loader are created by the Extension Technology utility: dxegenloadfile. Note that this utility will create both files. So, if you are using Data Express for z/OS, the referential integrity file created is used as input to the Import Classification from Referential Integrity job.

For more information, see the Getting Started with Distributed Data Stores guide.

Parent topic: Distributed Loader


Appendix A. Standard Mode

This appendix shows you how to prepare a working environment (or project) before using the Data Builder module.

The Standard configuration enables you to access data on your system in different ways. The database engine is Borland Database Engine (BDE) 5.01, which you install after you complete your Data Express installation on Windows.

Note: The Data Express for Distributed Systems solution is installed as standard mode. For Data Express for z/OS, you can install the product in standard mode if you choose.
  • Architecture
  • Database connection prerequisite
Parent topic: Front End Guide


Architecture

Standard Mode Architecture

BDE is inserted on another layer, the Borland SQL Links layer, which is installed together with BDE.

Connection to a database involves the creation of a database alias, which is a link to a database through a DB2 SQL Links database driver. The database alias is created from BDE Administrator.

Parent topic: Appendix A. Standard Mode


Database connection prerequisite

Before opening the project with Data Express, you must create a database alias that establishes a connection to the database.
  • DB2 database alias creation
  • Authorizations and privileges
Parent topic: Appendix A. Standard Mode


DB2 database alias creation

Before you can create a DB2 database alias, you must have an IBM Client Application Enabler, and you must have already cataloged all databases you want to access.
Notes:
  • The connection to the OS/390 system cannot be established through an ODBC DB2 database alias but only through a DB2 database alias (that is, only through SQL Links).
  • If you are using Data Express on Windows Vista and User Access Control is enabled, you must run the BDE Administrator utility as Administrator.

To create a DB2 database alias:

  1. From Windows Control Panel, open BDE Administrator.
  2. Click Object > New.
  3. In the Database Driver Name list, click DB2.
  4. Click OK to create the database alias.
  5. Enter the database alias name in the All Database Aliases pane.
  6. Click Object > Apply.
  7. In the Definition of pane, click in the second column for DB2 DSN to display a list of schemas for DB2 databases that have been cataloged on the PC.
  8. Select the appropriate database schema name, then click Object > Apply.
  9. Verify the DB2 SQL Links Driver uses the correct library to access DB2 Connect 5.2. On the Configuration page, select DB2 from /Configuration/Drivers/Native.
  10. Click in the second column for DLL32 and ensure that SQLDB2V5 is selected.
Parent topic: Database connection prerequisite


Authorizations and privileges

When you open a project, you must enter the user ID and password to connect to the database server where the project is located.

Data Express can save login information and use it for future connections to the same project, so you do not need to enter your user ID and password every time you connect. However, not saving login information is the default behavior.

The user ID used to establish the database connection must have read privileges on the following tables or views:

  • All tables created by Data Express in the MVS, OS/390, or z/OS DB2 environment during the installation phase.
  • SYSIBM.SYSTABLES
  • SYSIBM.SYSCOLUMNS
Parent topic: Database connection prerequisite


Appendix B. Client / Server Mode

This appendix shows how to configure access to the kbde-Server before using Data Express.

The actions listed in this section should be followed as a necessary first step before any serious use of Data Builder .

The Client/Server configuration provides access to the data resident on the MVS, OS/390, or z/OS system through the kbde-Server. The intermediate layer of this architecture is Borland MIDAS 3 and the connection to the server can be based on Socket, DCOM, or HTTP.

Note: You can only use client/server mode for Data Express for z/OS.
  • Architecture
  • Server connection prerequisite
Parent topic: Front End Guide


Architecture

Client Server Mode Architecture
Parent topic: Appendix B. Client / Server Mode


Server connection prerequisite

Before opening a project in Data Express, you must define connection information for the kbde-Server.
  • Defining server connection information
  • Authorizations and privileges
  • Starting Server-side Executables as Windows Services
    Explains how to configure your DB2 server to start server-side executables automatically as Windows services, enabling an automatic connection between the DB2 server and your Data Express modules.
Parent topic: Appendix B. Client / Server Mode


Defining server connection information

This section describes how to define server connections for the Data Builder and Data Subset Extraction modules.
Notes:
  • For more information about connection options to the kbde-Server, see Options.
  • Connection information is handled differently in the Data Masking module. For information about defining server connections for Data Masking, see the Data Masking Guide.

To define connection information:

  1. Click Tools > Options, then click the Server Information tab.
  2. Specify the following information:
    • IP Address – IP address of NT server (required for Socket and DCom).
    • Port Number – Port number assigned to Borland Socket Server (required for Socket).
    • URL – URL for the identification of the .dll installed on the server machine (required for ).
    • Connection options – Type of connection desired.
  3. Click OK to confirm.
Parent topic: Server connection prerequisite


Authorizations and privileges

Even though the connection to the MVS, OS/390, or z/OS system is established through the kbde-Server, authentication information is requested at the client level. In other words, when you connect to a project made available by the kbde-Server, you must enter a user ID and password to connect to the database server where the project is located.

Data Express can save login information and use it for future connections to the same project, so you will not need to enter your user ID and password every time you connect.

Note: Not selecting the Save user and password item from the Option window disables this feature.

The user ID used to establish the database connection must have read privileges on the following tables or views:

  • All tables created by Data Express in the MVS, OS/390, or z/OS DB2 environment during the installation phase.
  • SYSIBM.SYSTABLES
  • SYSIBM.SYSCOLUMN
Parent topic: Server connection prerequisite


Starting Server-side Executables as Windows Services

To establish a connection between Data Express modules and the Knowledge Base server, your DB2 server must be running the following server-side executables as services:

  • scktsrvr.exe - socket server
  • kbdeserver.exe - KBDE server

Data Express provides both executable files, which are located in the %ProgramFiles(x86)%\Micro Focus\Data Express 4.0\kbde-Server directory by default.

You can configure Windows to start these executables automatically as services each time you start Windows on the DB2 server. When both the socket and KBDE server services are started, the connection is established automatically.

When the socket server service is started on the DB2 server, Windows starts the KBDE server service automatically. Therefore, you need to configure only the socket server service to start automatically:

  1. Set scktsrvr.exe to run as an administrator:
    1. From Windows Explorer, change to the %ProgramFiles(x86)%\Micro Focus\Data Express 4.0\kbde-Server folder.
    2. Right-click scktsrvr.exe; then select Properties.
    3. Click the Compatibility tab.
    4. Check Run this program as an administrator; then click OK.
  2. At a Windows command prompt, change to the %ProgramFiles(x86)%\Micro Focus\Data Express 4.0\kbde-Server directory.
  3. Enter the following command:

    scktsrvr.exe /install

  4. Open Windows Services and start the Borland socket server, or reboot your machine.
Note: To remove these services:
  1. Stop the Borland socket server service.
  2. At a command prompt, change to the %ProgramFiles(x86)%\Micro Focus\Data Express 4.0\kbde-Server directory.
  3. Enter the following command:

    scktsrvr.exe /uninstall

Parent topic: Server connection prerequisite


Life Cycle Guide

Provides step-by-step instructions to help you handle modifications within the layout of the files already processed by Data Express.

The use of Data Express requires a minimum experience of the mainframe platform z/OS.

We recommend you carefully read the following users guides, which supplement this guide:

Guide | Description
Installation Guide | Explains installation procedures.
Data Model Guide | Describes the structure of the tables that comprise the Data Express Knowledge Base.
User Guide (z/OS) | Explains Data Express functions accessible from the z/OS system.
Front End Guide | Shows how to perform basic functions from the graphical front-end.
Data Masking Guide | Explains the Data Masking module.
Data Subset Extraction Guide | Explains how to create a new environment, starting from an existing environment and on the basis of selection criteria specified by the user.
  • Considerations
  • Life Cycle Introduction
  • Working with Life Cycle
  • Work with Life Cycle Jobs
Parent topic: Data Express


Considerations

Bear in mind the following points about Data Express:

  • To use this product, you must have the correct licence. See the Installation Guide for more details.
  • You do not need to analyze the sources of the programs in the application in order to use Data Express.
  • Data Express does not perform any operation capable of altering the analyzed data.
Parent topic: Life Cycle Guide


Life Cycle Introduction

Life Cycle is a tool within Data Express that keeps track of each modification applied to the file structure of your application, and prevents loss of information when these structural changes occur.

Note:
  • The Life Cycle is taken to mean exclusively the entering, into the database, of new data stores or modifications of the layout of existing data stores and not of modifications of the content that leave the file layout unaltered.
  • The algorithm used for the recognition of file modifications is based on the modifications of the sources (copybook COBOL and PL/I, SYSPUNCH, and DCLGEN) associated with the files. Direct-access DB2 files are the exception. The recognition of their modifications is based on the information contained in the DB2 system tables.
  • The Life Cycle feature of Data Express for z/OS also manages changes in the structure of distributed databases. To enable this feature, you use Distributed Loader to generate the IMPSRC file used as starting point for life cycle. For details see the Full Life Cycle Support topic in Getting Started with Distributed Data Stores.
Known Restriction
For DB2 tables with non-direct access, Life Cycle requires SYSPUNCH and DCLGEN to be partitioned data set (PDS) members, even though the cataloging phase supports SYSPUNCH and DCLGEN as sequential files. If you want to use Life Cycle and you have cataloged SYSPUNCH in a sequential file, you will need to delete it from the knowledge base and re-catalog it as a PDS member.
  • Life Cycle Phases
  • Sequential Steps
  • Concepts and Terms
Parent topic: Life Cycle Guide


Life Cycle Phases

In order to detect all the changes, the z/OS solution provides specific jobs that must be scheduled and executed. These operations are grouped into three phases:
  1. Changed Data Stores Identification (reading the IMPSRC file)
  2. Changed Data Elements Identification
  3. Apply Identified Changes
Note: For all phases, data stores are processed at the moment the phase is launched.
  • Phase 1: Changed Data Stores Identification
  • Phase 2: Changed Data Elements Identification
  • Phase 3: Apply Identified Changes
Parent topic: Life Cycle Introduction


Phase 1: Changed Data Stores Identification

Phase 1 of the Life Cycle, Changed Data Stores Identification, involves the following events:

  • Flat file population. Use the Toolkit to populate the IMPSRC, which contains a list of data stores and corresponding I/O areas that have been modified.
    Note: For more information, see Load IMPSRC for Life Cycle in your Toolkit (z/OS) guide.
  • File changes are identified. All copybooks included in the specified PDS or DB2 owner tables are scanned and compared with the structures already defined in the Data Express Knowledge Base.
  • A sequential guide file is created. This file is a list of information about the matches between the data stores included in the Data Express Knowledge Base and the copybooks, or of the modified DB2 tables.

    The guide file information is accessible in the Work with Changed Data Stores area.

Note: The guide file is followed by the IMPSRC file, which is created from the job Load Sequential for Life Cycle to recognize the modified sources. This Life Cycle feature of the toolkit can be omitted if the list is produced by your development management tool.

Before proceeding to the next phase, the list in the Work with Changed Data Stores area should be examined in order to confirm it and to possibly integrate additional information.

Parent topic: Life Cycle Phases
Related information
Phase 2: Changed Data Elements Identification
Phase 3: Apply Identified Changes


Phase 2: Changed Data Elements Identification

Phase 2 of the Life Cycle, Changed Data Elements Identification, involves the following events:

  • Field changes are identified. All copybooks included in the specified PDS or DB2 owner tables are scanned and compared with the structures already defined in the Data Express Knowledge Base.
  • A second guide file is created on the basis of the guide file generated during the previous phase. This file is a list of information about the matches between the data elements belonging to the file included in the Data Express Knowledge Base and the data elements of the copybooks or of the modified DB2 tables.
  • The guide file information is accessible in the Work with Changed Fields area.
Note: The recognition of the modified sources can be omitted if the list is produced by your development management tool.

Before proceeding to the next phase, this list in the Work with Changed Fields area should be examined in order to confirm it and to possibly integrate additional information.

Parent topic: Life Cycle Phases
Related information
Phase 1: Changed Data Stores Identification
Phase 3: Apply Identified Changes


Phase 3: Apply Identified Changes

Phase 3 of the Life Cycle, Apply Identified Changes, involves the following events:

  • Identified changes are applied.
  • File information, which is already contained in the Data Express Knowledge Base, is saved in an appropriate format. This includes sampling rules, associations of classes with fields, associations of the Data Changer with routines, test environment creation methods, and definitions of combined fields.
  • If required, a backup of the old data is made.
  • The modified data stores are analyzed again, and the previously saved information is applied to them, taking the new layouts into account.
Parent topic: Life Cycle Phases
Related information
Phase 1: Changed Data Stores Identification
Phase 2: Changed Data Elements Identification


Sequential Steps

The Life Cycle comprises several steps that are to be performed in the following order:

  1. Definition of environment:
    • Work with Companies - determine whether to enable the backup of a company and, if necessary, create a backup company.
  2. Definition of the Life Cycle phases:
    • Execute Phase 1 (Changed Data Stores Identification) - identify the PDS or schema (DB2 owner) on which the Life Cycle is to be launched. This phase consists of feeding in a guide file that contains a list of files and corresponding I/O areas that have been modified.
    • Execute Phase 2 (Changed Data Elements Identification) - create, on the basis of the guide file generated during the previous phase, a second guide file that contains matches between old and new fields.
    • Execute Phase 3 (Apply Identified Changes) - save, in an appropriate format, the information on files already contained in the Data Express Knowledge Base (for example, sampling rules, associations of classes with data elements, associations of masking routines, test environment creation methods, and definition of combined data elements), analyzes the modified data stores again and applies the previously saved information to them taking the new layouts into account.
  3. Modification of the guide files produced during the Changed Data Stores Identification and Changed Data Elements Identification phases in the following areas:
    • Work with Changed Data Stores
    • Work with Changed Fields
  4. Effective launch of phases:
    • Start Life Cycle Job
Parent topic: Life Cycle Introduction


Concepts and Terms

This section describes concepts and terms specific to the Life Cycle.

  • Backup Company
  • Log Files
Parent topic: Life Cycle Introduction


Backup Company

A backup company is useful when all the information about the files that follow the Life Cycle procedure needs to be preserved.

During the definition of a company, the backup company associated with it can be defined.

When the Life Cycle procedure is started on the files of a company associated with a backup company, the new file layouts will be visible within the company itself, whereas the information concerning the file with the layout considered to be obsolete may be retrieved from the backup company.

Note: Only one version for each file is stored in the backup company. Thus, if the Life Cycle causes a file to be modified twice, the backup company will only include the version before last and the version prior to that will no longer be stored.
Parent topic: Concepts and Terms
Related information
Log Files


Log Files

All the operations submitted from the client side are stored in log files that incorporate additional information including the submission date and time and whether or not the job was successful.

You can maintain log file information by using a specific function available on the client to delete entries (possibly starting from a given day).

Parent topic: Concepts and Terms
Related information
Backup Company


Working with Life Cycle

Describes the Life Cycle.

  • Definition of Environment
  • Execute Life Cycle Phase 1
  • Modification of Guide Files
Parent topic: Life Cycle Guide


Definition of Environment

This section describes the first step of the Life Cycle: Definition of Environment.

  • Work with Companies
Parent topic: Working with Life Cycle
Related information
Execute Life Cycle Phase 1
Modification of Guide Files


Work with Companies

To access the Work with Companies area, click [image] (Work with Companies) from the Life Cycle Phases window. An example of the Work with Companies window is shown below:

Figure 1. Work with Companies Window

Work with Companies Window

  • New Company Creation
  • Available Operations
Parent topic: Definition of Environment


New Company Creation

A new company can be created or an existing company can be modified or canceled from the Work with Companies window.

In addition, a company can be associated with information concerning the Life Cycle.

To create a backup company, just select the appropriate flag.

Warning: A backup company cannot be associated with more than one company.

While creating a standard company, you can decide whether to:

  • Enable the Life Cycle for all the files belonging to the company without creating any backup company. In this case, the flag related to the enabling of the Life Cycle must be selected and the fields related to the identification of a backup company must be left empty. This is the default behavior.
  • Inhibit the Life Cycle for all the files belonging to the company. In this case, no flag must be enabled.
  • Enable the Life Cycle for all the files belonging to the company, creating a backup company within which all the information about the layouts which existed before the Life Cycle is preserved. In this case, you need to select the flag related to the enabling of the Life Cycle and then a company from those proposed.
Parent topic: Work with Companies
Related information
Available Operations


Available Operations

The buttons at the bottom of the Work with Companies window are:

  • Apply - Confirms the changes made to the company. Before changes are applied, a series of checks on non-key fields are performed; if the checks fail, the update is blocked.
  • New - Initializes fields for insertion when creating a new company.
  • Delete - Removes the highlighted company.
  • Cancel - Cancels any changes made to the selected company. Any actions confirmed with the Apply button cannot be restored with Cancel but must be entered and confirmed again.
  • OK - Saves any changes that have not been applied and closes the window.
Parent topic: Work with Companies
Related information
New Company Creation


Execute Life Cycle Phase 1

Execute Life Cycle phase 1: Changed Data Stores Identification.

For more information about Life Cycle phases, see the chapter Work with life cycle jobs of the current manual and Life Cycle in the Process for z/OS Guide.

Parent topic: Working with Life Cycle
Related information
Definition of Environment
Modification of Guide Files


Modification of Guide Files

Describes the first step of the Life Cycle: Definition of Environment.

  • Work with Changed Data Stores
  • Execute Life Cycle Phase 2
  • Work with Changed Data Elements
  • Execute Life Cycle Phase 3
Parent topic: Working with Life Cycle
Related information
Definition of Environment
Execute Life Cycle Phase 1


Work with Changed Data Stores

After launching Phase 1 Changed Data Stores Identification, examine the list in the Work with Changed Data Stores area to confirm what Data Express has found, and possibly integrate it with additional information.

To access the Work with Changed Data Stores window, click [image] (Work with Changed Data Stores). An example of the Work with Changed Data Stores window is shown below:

Figure 1. Work With Changed Data Stores Window

Work With Changed Data Stores Window

This window shows the list of the copybooks or new/modified DB2 tables.

Select any of these sources on which you want to get more detailed information and carry out operations. The information is displayed in the grid columns and, in a summarized form, in the Life Cycle Guide Status section. The Data Element state box displays a status for the phase of the Life Cycle that has been carried out on the source involved.

To delete life cycle information for the specified data store, right click an element of the list and choose Delete. If life cycle is not completed for the data store, the delete operation will make life cycle ignoring the changes for the deleted data store.

These fields can be modified by clicking the Change button: the value might change from Executed to Not Executed in case you want to re-execute the phase to which the field refers.

When a confidence field assumes a Not recognized value, it indicates whether Life Cycle needs additional information or user confirmation. Only when the field assumes the Recognized value, can the following phase be carried out.

The possible cases are outlined below:

  • New copybook is not associated to file. Even the case in which the source is new but refers to an additional format/segment of an existing file falls within this category.
  • Existing copybook is associated with existing file.
  • Existing copybook is not associated with any file.
  • New copybook is associated with new data store (for DB2 files only).
Note: If a copybook defines several file layouts and only one of these has changed, Data Express identifies all the data stores (including the non-modified data stores) as changed data stores. At the end of the Life Cycle, only the modified data stores will contain the modifications made.
  • Case 1 - New copybook is not associated with data store
  • Case 2 - Existing copybook is associated with an existing data store
  • Case 3 - Existing copybook is not associated with any data store
  • Case 4 - New copybook is associated with new data store (for DB2 files only)
Parent topic: Modification of Guide Files
Related information
Execute Life Cycle Phase 2
Work with Changed Data Elements
Execute Life Cycle Phase 3


Case 1 - New copybook is not associated with data store

If Phase 1 identifies a new copybook and Data Express cannot determine which data store the copybook should be associated with, you are asked to specify further information.

In this case, you should always specify an existing machine ID and a company name (not a backup company) for the copybook. There are three ways to make the confidence field take on the Recognized value:

  • Leave the copybook unassigned - No other operation is required.
  • Set an existing data store - A data store is specified from the list of data stores that already exist in the Knowledge Base, which is made available by clicking the New Data Store. The copybook is then associated with the selected data store.
  • Specify a new data store - A new data store is specified, also entering all the information required to load it. The copybook is then associated with the new data store specified.
Parent topic: Work with Changed Data Stores
Related information
Case 2 - Existing copybook is associated with an existing data store
Case 3 - Existing copybook is not associated with any data store
Case 4 - New copybook is associated with new data store (for DB2 files only)


Case 2 - Existing copybook is associated with an existing data store

Phase 1 has detected that an existing copybook assigned to a data store has been modified. In this case, you cannot change anything about the copybook except for the data store assignments.

  • Leave the copybook assigned to the data store - No other operation is required.
  • Set an existing data store - A data store is specified from the list of data stores that already exist in the Data Express Knowledge Base, which is made available by clicking the New Data Store button. The copybook is then associated with the selected data store.
  • Specify a new data store - A new data store is specified, also entering all the information required to load it. The copybook is then associated with the specified data store.
Parent topic: Work with Changed Data Stores
Related information
Case 1 - New copybook is not associated with data store
Case 3 - Existing copybook is not associated with any data store
Case 4 - New copybook is associated with new data store (for DB2 files only)


Case 3 - Existing copybook is not associated with any data store

Phase 1 has detected that an existing copybook has been modified but is not associated with any data store. This may indicate that the previously modified copybook had not been assigned to any data store or that the copybook has been modified to such an extent that Data Express is no longer able to reconstruct its association with the data store.

In this case, you cannot change anything about the copybook but can assign another data store or leave it unassigned.

  • Leave the copybook unassigned - No other operation is required.
  • Set an existing data store - A data store is specified from the list of data stores that already exist in the Knowledge Base, which is made available by clicking the New Data Store button. The copybook is then associated with the selected data store.
  • Specify a new data store - A new data store is specified. (You must also enter all the information required to load it.) The copybook is then associated with the new data store specified.
Parent topic: Work with Changed Data Stores
Related information
Case 1 - New copybook is not associated with data store
Case 2 - Existing copybook is associated with an existing data store
Case 4 - New copybook is associated with new data store (for DB2 files only)


Case 4 - New copybook is associated with new data store (for DB2 files only)

Phase 1 has detected a new DB2 file. In this case, you must specify an existing machine ID and a company name.

Parent topic: Work with Changed Data Stores
Related information
Case 1 - New copybook is not associated with data store
Case 2 - Existing copybook is associated with an existing data store
Case 3 - Existing copybook is not associated with any data store


Execute Life Cycle Phase 2

Execute Life Cycle phase 2: Changed Data Elements Identification:

For more information about Life Cycle phases, see the chapter Work with life cycle jobs of the current manual and the chapter Life Cycle in the Process for z/OS Guide.

Parent topic: Modification of Guide Files
Related information
Work with Changed Data Stores
Work with Changed Data Elements
Execute Life Cycle Phase 3


Work with Changed Data Elements

The Changed Data Elements Identification phase performed should be confirmed and/or modified from the Work with Changed Fields window. The batch job identifies matches between the old copybook data elements. The processing operation might miss something (the value assumed by the confidence field is different from Recognized), so you must force the confidence field value to Recognized.

To access the Work with Changed Fields window:

  1. Click Work with Changed Data Stores (Work with Changed Data Stores) from the Life Cycle Phases window.

    The Work with Changed Fields window is displayed:

    Figure 1. Work With Changed Data Stores Window

    Work With Changed Data Stores Window

  2. Select the required row from the List of Changed Data Stores and click the Changed Data Elements button. An example of the Work with Changed Fields window is shown below:
    Figure 2. Work With Changed Fields Window

    Work With Changed Fields Window

    The two upper lists show the old and new data elements list for the selected source. Within the two lists, the different icons alongside the listed fields identify data elements that have been matched and data elements that have no matches.

    The lower list shows the matches between the new and old data elements.

    Note: If there are no matched data elements, this list will be empty.
  3. Identify new matches between data elements and define those that Data Express could not recognize.
  4. If required, modify or delete the matches identified by Data Express.
    Note: Bear in mind that the following steps only apply to the records of the list of confirmed data elements.
  5. Select the two data elements to be compared from the upper lists, and click the Match Data Elements button to view matches in the lower list.
    Note: To delete a match between data elements, select it from the lower list and click the Delete Match button.

Data Express automatically recognizes some relationships. Therefore, if only one data element has been added to a table, leaving everything else as it was, all the data elements will appear to be matching. The new data element is displayed on the right-hand side and no matching is needed. This form is used to specify the data elements that have changed, for example, whenever a data element has been radically changed and the z/OS platform application cannot recognize it.

Data Express is designed to handle the following:

  • data elements to be added (at the head, tail, or in the middle)
  • data element to be split up into several data elements
  • several data elements to be combined in a single data element (combined data element)
  • data elements to be eliminated
  • data element typology to be changed (type/length/decimals)

To identify matches between old and new data elements, Data Express mainly relies on the correspondence of:

  • data element name
  • position (the order number of the data elements) within the data store
  • data element type and length
Parent topic: Modification of Guide Files
Related information
Work with Changed Data Stores
Execute Life Cycle Phase 2
Execute Life Cycle Phase 3


Execute Life Cycle Phase 3

Execute Life Cycle phase 3: Apply Identified Changes.

For more information about Life Cycle phases, see the chapter Work with life cycle jobs of the current manual and the chapter Life Cycle in the Process for z/OS Guide.

Parent topic: Modification of Guide Files
Related information
Work with Changed Data Stores
Execute Life Cycle Phase 2
Work with Changed Data Elements


Work with Life Cycle Jobs

Life Cycle jobs can be scheduled from the Work with Jobs area.

For more information about the Work with Jobs area, see the Work with Jobs in the Front End Guide.

  • Introduction
  • Life Cycle Job Creation
  • Clear Life Cycle History Job Creation
  • Available Operations
Parent topic: Life Cycle Guide


Introduction

To access the Work with Jobs area:

  1. Start Data Builder by clicking Start > All Programs > Micro Focus Data Express 4.0 > Data Builder.
    Note: If you are using Data Express on Windows Vista and User Access Control is enabled, you must run Data Builder as Administrator.
  2. Click [image] (Work with Jobs) in the toolbar of the main window.

    The Work With Jobs area is displayed as shown below:

    Figure 1. Work with Jobs Area

    Work with Jobs Area

Parent topic: Work with Life Cycle Jobs
Related information
Life Cycle Job Creation
Clear Life Cycle History Job Creation
Available Operations


Life Cycle Job Creation

The Life Cycle job executes each phase of the Life Cycle sequentially. Each phase is executed until confirmation is needed.

Note: Using the Life Cycle job to execute the Life Cycle phases does not take into consideration the list of scheduled phases (displayed in the Life Cycle Phases window) or the time limit parameter.

To create a Life Cycle job:

  1. From the Work with Jobs window, click New.
  2. In the List of Jobs section, select BLCFILR - Life Cycle.
  3. Click Apply to submit the job. The Secondary Options window appears:
    Figure 1. BLCFILR - Life Cycle Secondary Options

    BLCFILR - Life Cycle Secondary Options

  4. In the Secondary Options window, select one or more phases according to your requirements.
  5. Click OK.
Parent topic: Work with Life Cycle Jobs
Related information
Introduction
Clear Life Cycle History Job Creation
Available Operations


Clear Life Cycle History Job Creation

This job provides a deletion of table ANLICLOG. This table is the life cycle log. It is for internal use only and it is not shown in any screen of the user interface.

To create a Clear Life Cycle History job:

  1. From the Work with Jobs window, click New.
  2. In the List of Jobs section, select the Clear Life Cycle History job.
  3. In the General section, select the appropriate Machine IdandCompany name.
  4. Click Apply.

    The Secondary Options window appears:

    Figure 1. BLCCLR - Clear Life Cycle History Secondary Options

    BLCCLR - Clear Life Cycle History Secondary Options

  5. In the Secondary Options window, specify values for the following fields:
    • Remove History Before - Removes all Life Cycle information before the selected date.
    • Reorganize Table - Performs a reorganization of the table after removing the Life Cycle information.
Parent topic: Work with Life Cycle Jobs
Related information
Introduction
Life Cycle Job Creation
Available Operations


Available Operations

The buttons in the List of Scheduled Jobs section of the Work with Jobs window and the Secondary Options window are:

  • Change - Lets you modifyFile Qualification parameters for the selected job if the job is scheduled but has not yet been submitted (Flag Status is 0). Changes should be saved using the Apply button.
  • Apply - Saves the information for the new job, but keeps the window open.
  • New - Initializes fields for insertion when scheduling a new job.
  • Delete - Cancels the execution of the selected scheduled job and removes the job from the List of Scheduled Jobs list. If the job is currently being executed (Flag Status is 1), you will be prompted to confirm the deletion.
  • Cancel - Cancels any changes. Any confirmations made with the Apply button cannot be undone with the Cancel button but must be instead entered and confirmed again.
  • OK - Applies all changes and closes the window.
  • Reset All - Clears the values of all fields.
Parent topic: Work with Life Cycle Jobs
Related information
Introduction
Life Cycle Job Creation
Clear Life Cycle History Job Creation


Process for z/OS Guide

Outlines the concepts and procedures used by Micro Focus Data Express for z/OS. It shows how to start and work with Data Express in the mainframe environment.

Data Builder is a general-purpose data analysis module for z/OS environment. The module comprises a part on the z/OS platform and one on a PC. All operations and analysis are performed in the z/OS environment. All the analysis results are stored in a DB2 repository.

Who Should Read this Guide

This guide is intended for z/OS users who want to verify the integrity of data in databases (for example, DB2 or DL/I) or data sets (for example, VSAM, SEQUENTIAL, or GDG) of applications running on z/OS that are either developed internally or purchased by external suppliers.

The use of Data Express requires a minimum experience of the mainframe platform z/OS.

Considerations

Bear in mind the following points about Data Express:

  • To use this product, you must have the correct licence. See the Installation Guide for more details.
  • You do not need to analyze the sources of the programs in the application in order to use Data Express.
  • Data Express does not perform any operation capable of altering the analyzed data.
  • Project Implementation
    Describes what you need to do to start and work with Data Express.
  • Life Cycle
    This chapter explains the day-to-day activities to update Data Express with the normal changes that occur to the files, tables, and copybooks in the user environment.
  • Image Copy
    Data Express manages image copies of DB2 tables in the Data Masking and Data Subset Extraction modules.
  • Starting Data Express for z/OS
    How to start Data Express
  • Submission Function
    Data Express lets you submit scheduled jobs automatically from the Data Builder module so that you can work from your PC to schedule jobs to verify the results of executed jobs.
  • GDG Support
    Describes the correct format for specifying the names of the GDG generations.
Parent topic: Data Express


Project Implementation

  • Data Inventory
    The Data Inventory process lets you identify the structure of your data store and its associated characteristics through various forms of analysis and associations.
  • Data Classification
    Data Express groups fields automatically into homogeneous categories (or classes), on the basis of both their contents "snapshot" and their format and naming convention.
  • DB2 Catalog Synchronization
    The job DB2 Catalog Synchronization loads information for the cardinality for DB2 tables and the existence of a key or the existence of an image copy into the Data Express Knowledge Base.
  • Data Subsetting
    The Data Subsetting function enables the creation of a new reduced environment starting from an existing environment using selection criteria specified by the user.
  • Data Masking
    About data masking routines
Parent topic: Process for z/OS Guide


Data Inventory

The information about your data store that is loaded into the Data Express Knowledge Base is as follows:
  • Data store (or table) inventory
  • Type classification (for example, DB2, SEQ, or VSAM)
  • Record number and record length
  • Data store (or table) structure
  • •Application structure

For non-relational data stores, the application structure of the Data Inventory process is organized into two phases:

  • Load and analysis of the copybook that describes the structure of the data stores
  • Load, analysis, and association of the data store
For relational data stores (for example, DB2), the application structure of the Data Inventory process is organized into one phase:
  • Load DB2 tables
For ADABAS data stores, the application structure of the Data Inventory process is organized into one phase:
  • Load ADABAS tables
  • Load Copybook Information
  • Load DB2 Tables
  • Load ADABAS Data Stores
  • Load Other Data Stores
Parent topic: Project Implementation


Load Copybook Information

The load and analysis of the copybook can be done using the job Load Copybook from External Interface (see the sections Load Copy Information From External Interface Job in the chapter Work with Jobs in the Front End Guide and Load Data Store Information from External Interface in the chapter Sequential Files in theData Model Guide.)

Parent topic: Data Inventory


Load DB2 Tables

For DB2 tables with direct access, you can use the job Load Data Store Information from External Interface to load these tables into the Data Express Knowledge Base (see the section Load Data Store Information From External Interface Job in the chapter Work with Jobs in theFront End Guide).

All the information concerning the structure of the DB2 tables must be in a sequential file that will be processed to find the inventory information needed (see the chapter Load Data Store Information from External Interface Sequential File Job in the Data Model Guide).

For information about the sequential file used as input for the job Load Data Store Information from External Interface, see the chapter Load Sequential File with DB2 Catalog in the Toolkit (z/OS).

Parent topic: Data Inventory


Load ADABAS Data Stores

Data Express does not support a direct link to the ADABAS catalog. Instead, you must use the ADABAS FDT Report to generate an output sequential file that details the structure of the ADABAS tables. This sequential file is then used to load your ADABAS data store information into the Data Express Knowledge Base.

However, in order to load the table information into the KB, you must specify the output sequential file while submitting the job Load Data Store Information from External Interface (see the section Load Data Store Information From External Interface Job in the chapter Work with Jobs in the Front End Guide.)

For information about the sequential file that is used as input for the job Load Data Store Information from External Interface, see the chapter Load Sequential File with ADABAS FDT report in the Toolkit (z/OS) guide.

For information about the structure of the sequential file, see the section Load File Information from External Interface from chapter Sequential File in the Data Model Guide.

Parent topic: Data Inventory


Load Other Data Stores

Loading all the other types of data stores into the Knowledge Base can be done by executing the following:

  • Job Load Data Store Information from External Interface (see the section Load Data Store Information From External Interface Job in the chapter Work with Jobs from theFront End Guide).
  • Single load job for all the other types of data store including VSAM or GDG (see the sections Load VSAM data set information Job, Load SEQ/GDG Data Set Information Job, Load DL/I Database Information Job, and Load DB2 table information by unload Job in the chapter Work with Jobs in the Front End Guide).
Note: When using VSAM RRDS data sets, you must specify RRDS in the interface instead of a generic VSAM term. In addition, you must create a process ID similar to GENDA for RRDS data sets, but use I/O program KURRDS instead of KURFIO.

For the DB2 tables with unload, the inventory information is located in the SYSPUNCH, while for the other types of data stores, the inventory information is located in the copybook associated with them.

The copybook association can be manual or automatic:

  • Manual (see the section Load Copybook Information from External Interface Job in the chapter Work with Jobs from theFront End Guide).
  • Automatic with the job Load Data Store Information from External Interface (see the section Load Data Store Information from External Interface in the chapter Sequential Files from the Data Model Guide).
Parent topic: Data Inventory


Data Classification

Data Express groups fields automatically into homogeneous categories (or classes), on the basis of both their contents "snapshot" and their format and naming convention.
Data classification can be performed in four different ways:
  1. Using the job Import Classification from Referential Integrity
  2. Using the job Import Classification from Data Dictionary
  3. Through manual association
  4. Through Data Analysis
  • Import Classification from Referential Integrity Job
  • Import Classification from Data Dictionary Job
  • Manual Association
  • Data Analysis
Parent topic: Project Implementation


Import Classification from Referential Integrity Job

The job Import Classification from Referential Integrity lets you import the data classification from a CSV file with Referential Integrity relations (see the sections Import Classification From Data Dictionary Job in the chapter Work with Jobs in theFront End Guide and Load Referential Integrity Relation Information in the chapter Sequential Files in the Data Model Guide). This job can be used for data subsetting. This job sets the estimated class assignment, which can be confirmed in the Work with Fields area of Data Builder (see the chapter Work with Fields in theFront End Guide).

Note: If you are using Data Express for z/OS with the optional ODBC Extension, you can specify your ODBC DSN in the ODBC DSN Name field, provided that it is less than 44 characters in length.
Parent topic: Data Classification


Import Classification from Data Dictionary Job

The job Import Classification from Data Dictionary lets you import the data classification from a sequential file with the list of the field and the related class that must be associated with it (see the chapter Import Classification From Data Dictionary Job in the Front End Guide). This job can be used for data masking.

Note: If you are using Data Express for z/OS with the optional ODBC Extension, you can specify your ODBC DSN in the ODBC DSN Name field, provided that it is less than 44 characters in length.
Parent topic: Data Classification


Manual Association

Manual association lets you associate a class with a specific field or a list of fields using Data Builder (see the section Window Contents in the chapter Work with Classes in the Front End Guide).

Parent topic: Data Classification


Data Analysis

In the same way a fingerprint is used to identify a person, the fingerprint is a way to identify a field by its characteristics. The base concept is that different kinds of fields have different types of content and therefore have different fingerprints.

The fingerprint is calculated by the Sampling function, which displays concisely the content of the data store data elements or a sample of the values contained in the fields with the number of occurrences for each of them.

After having calculated the fingerprint, if you know that the corresponding data element belongs to a specific class of data element (for example, customer code), you can use the fingerprint of this data element as a prototype, and assign it to this class.

The next step is to search all the data elements that have a fingerprint similar to the prototype assigned to the class with the job Class Data Element Assignment (see the chapter Class Data Element Assignment Job in the Front End Guide). This job suggests a class that is assigned a level of confidence calculated using recognition algorithms based on the compatibility of definition attributes and similarity of contents.

After job execution, the suggested class can be confirmed (see the section Estimated ClassesConfirm Tab in the chapter Work with Data Elements in the Front End Guide.)

Parent topic: Data Classification


DB2 Catalog Synchronization

The job DB2 Catalog Synchronization loads information for the cardinality for DB2 tables and the existence of a key or the existence of an image copy into the Data Express Knowledge Base.

The job must be executed before subsetting the data using Data Subset Extraction. If cardinality information or the existence of a key or image copy stored in the system catalog has changed, the job must be executed again.

For more information, see chapter DB2 Catalog Synchronization in the Toolkit (z/OS).

Parent topic: Project Implementation


Data Subsetting

The Data Subsetting function enables the creation of a new reduced environment starting from an existing environment using selection criteria specified by the user.

The application structure of the Data Subsetting function is organized in two phases:

  1. Creation of methods to generate a test environment
  2. Submission of extraction jobs (see the chapter Phase 4 Launch of the Extraction Jobs in the Data Subset Extraction Guide).
  • Method Creation
  • Work with Method Function
  • Extraction Jobs Submission
  • Output Flat File
Parent topic: Project Implementation


Method Creation

A method can be created by:

  • The job Import Method from Referential Integrity (see the chapter Import Method From Referential Integrity Job in the Front End Guide). The method is created starting from a data store for a specific class that includes all the related files indicated in the CSV file that contains the Referential Integrity relation.
  • The Data Subset Extraction wizard (see the chapter Create New Method Wizard in theData Subset Extraction Guide). Starting from a class or from a related class, the wizard guides you in creating a method with all the data stores that have the specified classes; with this wizard, you can define the logical model of data, define the rules of subsetting, and add or remove files and specify the input and output data set for files involved in the method.
  • The Data Masking module. This module can be used only if the new environments are created only applying data masking techniques and all the data stores will be expanded separately (see the Data Masking Guide ).
Parent topic: Data Subsetting


Work with Method Function

After the creation of the method, it can be edited in the "Work with method" function. The following list describes the capabilities of this function:

  1. Add or exclude a data store from the method.
  2. Change the output file.
  3. Modify the filter (reduction criterion) for a data store (or for all data stores with the same filter).
  4. Specify one or more output classes for a data store (if they are not generated during the initial method creation) using the same values for the pairs: (selection class, input filter) and (out class, output filter). In order for an output class to be used as an input class for another data store, during the processing of the data store (in a table of the Knowledge Base called ANENVLST) the values for the data element containing the output class are created.
    Note: In this example, two entries are generated for the same data store with all the same values except for the output class. During the extraction, all the entries for the same data store are processed together, and the class information is stored in the table ANENVLST for each output class.
  5. Specify one or more input classes (multiple filters) for a data store. In this case, an OR condition is implicitly generated by Data Express. (If you want to use an AND condition, you should create a combined data element and specify a single condition on the data element.)

See the chapter Work with Method Multiple Filters in the Data Subset Extraction Guide.

Parent topic: Data Subsetting


Extraction Jobs Submission

During the extraction job, all the data stores that have data elements with a data masking routine associated with them will be automatically masked.

The Data Subsetting process manages all types of data stores processed by Data Builder with the following exceptions:

  • DB2 with direct access residing in a different DB2 subsystem from the one in which the product is installed
  • DL/I file with direct access

In these two cases, before creating the extraction method, you must specify a sequential unload of the files and insert the Process Identifier for reading the files, as described in the section Specifying Unload Files for DL/I and DB2 Direct Access Files in the chapter Data Subset Extraction Creation in the Data Subset Extraction Guide.

Parent topic: Data Subsetting


Output Flat File

For all files involved in the subsetting, the product output is sequential files in the same format of the input. Depending on the mode used to create the method, there are different ways to indicate this output flat file.

If the method was not created (as the new environments must be created only applying data masking techniques), the name of the flat file must be indicated for all the files to be expanded.

Import Method from Referential Integrity

If the method is created by the job Import Method from Referential Integritythe name of the output flat file and the routine used to write it can be specified using Data Subset Extraction. In this case, you must manually update this information in the Elaboration properties windows for all the files involved in the method (see the chapter Work with MethodElaboration Properties in the Data Subset Extraction Guide ). Otherwise, this operation can be done with a query that directly updates the DB2 table HSDCHFIL that contains this information. For example, the following query can be used:

UPDATE EXN01.HSDCHFIL A
SET WRTPGM = 'KDCWRT', UNLOUTNAM = (SELECT 'HAL.KBAIMD.UNLOAD.' CONCAT TSNAME
FROM EXN01.HSURDFIL B
WHERE A.MCRECID = B.MCRECID AND A.FILRECID = B.FILRECID)
WHERE METHOD = 'METHOD4'

This query updates the name of the routine (field WRTPGM) and the flat file (UNLOUTNAM) for the DB2 tables. The name of the flat file is based on a string to which is added the name of the tablespaces (present in the field TSNAME of the table HSURDFIL: this table contains all the information concerning data stores loaded into Data Express). It is assumed that the names of the tablespaces are always different in order to obtain different flat files. However, if this is not so, this query can be customized for your user environment.

For a sequential file, add a string to the file name as shown in the following query:

UPDATE EXN01.HSDCHFIL A
SET WRTPGM = 'KDCWRT', UNLOUTNAM = (SELECT 'HAL.KBAIMD.UNLOAD.' CONCAT FILNAME
FROM EXN01.HSURDFIL B
WHERE A.MCRECID = B.MCRECID AND A.FILRECID = B.FILRECID)
WHERE METHOD = 'METHOD4'

For both previous queries, the owner of the tables HSDCHFIL and HSURDFIL must be changed in accordance with the parameters specified during the installation of Data Express. Moreover, instead of using the string 'METHOD4' in the field METHOD, you must indicate the name of the method to be executed, and instead of using the string 'HAL .KBAIMD.UNLOAD.' you can indicate another string with the initial part of the output flat file.

Data Extraction Wizard

If the method is created with the Data Subset Extraction wizard, the name of the flat file and the name of the routine used to write it are automatically inserted in the table HSDCHFIL.

The name of the flat file can be generated following one of these three rules:

  1. A prefix is added to the unload data set name, or, if it doesn't exist, to the file name.
  2. A suffix is added to the unload data set name, or, if it doesn't exist, to the file name.
  3. A string replaces a part of the unload data set name or, if it doesn't exist, the file name.

One of these three rules generates a consistent flat file name for all files except the DB2 tables loaded with direct access. For this kind of file, all these three rules would produce a name with the wrong syntax. So, as the DB2 tables with direct access do not have the unload data set name, you can update the field UNLFILNAM in the table HSURDFIL before creating the method. For example, you can use the following query:

UPDATE EXN01.HSURDFIL A
SET UNLFILTYP = 'SEQ', UNLFILNAM = CONCAT ('HAL.KBAIMD.UNLOAD.', TSNAME)
WHERE UNLFILNAM =' AND FILTYPE = 'DB2'

Also in this query, the owner of the table HSURDFIL must be changed in accordance with the parameters specified during the installation of Data Express, and instead of the string 'HAL .KBAIMD.UNLOAD.you can indicate another string that will be the initial part of the output flat file.

Parent topic: Data Subsetting


Data Masking

This process allows for the creation of a new environment, starting from an existing environment and based on masking routines that must preserve:

  • the original meaning of the information
  • data referential integrity
  • the unique key

Data Express provides a standard masking routine for:

  • NamesSurnames
  • Company names
  • Addresses
  • Unique alphanumeric codes
  • Telephone numbers

However, you can develop customized masking routines for each standard case (for example, compress data or data with control characters) using standard programming languages (COBOL, PL/I, or C).

The application structure of Data Masking is organized in two phases:

  1. Creation of methods to generate a test environment (see the chapter Create New MethodWizard in the Data Subset Extraction Guide). This phase is not mandatory in that if the method is not created; all the files are expanded separately.
  2. Submission of masking jobs (see the chapter Work with Data Changer Jobs in theData Masking Guide).
Parent topic: Project Implementation


Life Cycle

This chapter explains the day-to-day activities to update Data Express with the normal changes that occur to the files, tables, and copybooks in the user environment.

The Life Cycle process manages modifications within file layouts, preserving any manual information already entered into the Data Express Knowledge Base.

The application structure of Life Cycle is organized in three phases:

  • Phase 1: Changed Data Stores Identification
  • Phase 2: Changed Data Elements Identification
  • Phase 3: Apply Identified Changes

For known restrictions pertaining to the Life Cycle process, see chapter Life Cycle Introduction in the Life Cycle Guide

  • Phase 1
  • Phase 2
  • Phase 3
Parent topic: Process for z/OS Guide


Phase 1

Phase 1, starting from a list of copies or of new or modified DB2 tables, verifies if they are already in the Data Express Knowledge Base and then produces the list of data stores seen in the Work with Changed Data Stores area.

The list of copies or DB2 tables from which phase 1 starts must be in a sequential file named like the project and group specification and must be followed by the suffix IMPSRC. (For the structure of this file, see chapter IMPSRC sequential FileModified Source in the Data Model Guide.) The file can be produced by:
  • The Data Express Toolkit (See the chapter Load IMPSRC for Life Cycle in the guide Toolkit (z/OS)).
  • The user with a change management system.

Before moving on to the next phase, you must examine the list in the Work with Changed Data Stores area to confirm what Data Express has found, and possibly integrate it with additional information. (See the section Work with Changed Data Stores in the Life Cycle Guide.)

Parent topic: Life Cycle


Phase 2

Phase 2 starts from the list of copies or new or modified DB2 tables present in the Work with Changed Data Stores area that are correctly expanded by phase 1 (that is only the record with File Confidence = Recognized) and compares their current structure with what is already defined in the Data Express Knowledge Base to verify if there are matches between their data elements.

At the end of this phase, a list of changed data stores and data elements is produced, which you can verify and approve. Thanks to a confidence parameter, you can identify where the tool could recognize the current features automatically, where a manual confirmation of what has been found is required, or where the match between the original and the modified features must be set manually.

Before moving on to the next phase, you must examine the list in order to confirm what the product has found and, possibly, integrate it with additional information. (See the chapter Work with Changed Data Stores of the Life Cycle Guide.)

Parent topic: Life Cycle


Phase 3

Phase 3 applies all the changes identified in the previous phases and, if required, makes a backup of the old data. This phase also expands all copies or DB2 tables that were correctly expanded by phase 2 (that is only the record with Field Confidence = Recognized and Field State = Executed).

All the information already contained in the Data Express Knowledge Base (for example, sampling rules, associations of classes with data elements, associations of the Data Changer function with routines, test environment creation methods, and definition of combined data elements) are saved and applied to the new structure of the file.

A backup project is useful when all the information about the files that undergo the Life Cycle procedure needs to be preserved.

During the definition of a project, you can define the backup project associated with it. When the Life Cycle procedure is started on the files of a project associated with a backup project, the new file layouts will be visible within the project itself, whereas the information about the file with the layout considered to be obsolete may be retrieved from the backup project.

Note: Only one version for each file is stored in the backup project. Thus, if the Life Cycle causes a file to be modified twice, the backup project will only include the version before last whereas the version prior to that will no longer be stored.
Parent topic: Life Cycle


Image Copy

Data Express manages image copies of DB2 tables in the Data Masking and Data Subset Extraction modules.

The DB2 tables must be loaded into the Knowledge Base with direct access. Then, during the masking or subsetting process, you can get the data from the backup obtained by image copy instead of the production table.

To obtain information from the image COPY, create a process identifier with the appropriate access type as described in the Front End Guide. In the following example, we assume the process identifier is DB2IC.

Before beginning the extraction or masking phase, you must execute the synchronization phase, to load the information concerning the image copies into Data Express.

Then, the name of the image copies data set must be inserted in the field UNLINPNAM of the data store HSDCHFIL (This information is specified in the File Name data element in the Unload Input File Properties section of the Elaboration properties window of the Work with Method area in Data Subset Extraction ) so the process can read the correct image copy as input file. (For more information, see the section Window Contents in the chapter Work with MethodElaboration Properties in the Subset Extraction Guide.)

To do this, use the exit routine UTERTIC (that must be customized to obtain the name of the image copies matching your environment) or use a query such as the following:

UPDATE KBAIMD.HSDCHFIL A
SET UNLINPTYP = 'SEQ', PROCIDINP = 'DB2IC', UNLINPNAM = (
SELECT DSNAME FROM KBAIMD.HSURDICP B, KBAIMD.HSURDFIL D
WHERE D.MCRECID = 1
AND D.FILRECID = A.FILRECID
AND D.TSNAME = B.TSNAME
AND D.DBNAME = B.DBNAME
AND B.DSNUM < 2
AND B.ICDATE = '050319')
WHERE A.MCRECID = 1
AND A.METHOD = 'PRIVACY1'
AND EXISTS (
SELECT DSNAME FROM KBAIMD.HSURDICP C, KBAIMD.HSURDFIL E
WHERE E.MCRECID = 1
AND E.FILRECID = A.FILRECID
AND E.TSNAME = C.TSNAME
AND E.DBNAME = C.DBNAME
AND C.DSNUM < 2
AND C.ICDATE = '050319')

This query must be updated to change the owner of the tables HSDCHFIL, HSURDICP, and HSURDFIL in accordance with the parameters specified during the installation of Data Express.

Additionally, instead of using the value '050319' in the fields ICDATE, you must indicate the date of the image copy from which the process must read the data; instead of 'PRIVACY1' in the field METHOD, you must indicate the name of the method that must be executed, and instead of '1' in the field MCRECID, the progressive number associated with the Machine ID and the Company present in the method must be indicated.

After inserting the name of the image copy, the job Create Data Changer (for data masking) or the job Create Extraction Job (for data subsetting) can be executed. Both these jobs automatically generate a JCL that, starting from the image copy, unloads the information about the DB2 tables and the data into a temporary flat file that will be deleted at the end of the job. After this first step, the masking or subsetting program is submitted.

Restriction: The jobs Test Environment Creation and Data Store Data Changer cannot be used on files with access from image copy.

A similar process can be applied to the backups of the DB2 tables obtained by the unload utility of CA in VARIABLE, FIX, or COMMA SEPARATED format.

Parent topic: Process for z/OS Guide


Starting Data Express for z/OS

The Data Express for z/OS client runs on z/OS systems.

To start Data Express for z/OS:
  1. Either execute the following command from a TSO environment:
    EXEC 'xxxxxxxx.yyyyyyyy.CLIST(MFDATA)'
    or execute the following command from an ISPF command line:
    TSO EXEC 'xxxxxxxx.yyyyyyyy.CLIST(MFDATA)'

    where xxxxxxxx.yyyyyyyy is a part of the PDS library name where member MFDATA is located. In both cases, this command allocates PDS libraries to ISPF and launches the Main Menu in an ISPF environment as shown below:

  2. Specify the appropriate option from one of the three sections as listed below.
    Note: If this is the first time you have started Data Express on z/OS, you must first enable your product. For more information, see the section Product Enabling.
    Submission Function
    Lists an option to submit scheduled jobs automatically for execution.
    Product Enabling
    Lists options to retrieve and specify the software key, and check product licensing status.
    Environment Creation and PTF
    List options to complete the product installation and to install both single PTFs and a whole modification level. For more information, see theInstallation Guide.
Parent topic: Process for z/OS Guide


Submission Function

Use the Submit Client Schedule Job panel to set options for job scheduling and execution.

For more information about scheduling jobs, see the chapter Work with Jobs in the Front End Guide

Restriction: You cannot use Data to schedule Data Store Data Element Sampling jobs for direct access DL/I databases.

This section describes the Submit Client Scheduled Job (BURSUBM) panel.

The client scheduled job reads the Knowledge Base to determine if a new job has been created in the client Work with Jobs area, and then submits the jobs accordingly. The client scheduled job is also called the submitter job.

To submit client scheduled jobs:
  1. From the Main Menu panel, select the option Submit Client Scheduled Jobto display theSubmit Client Scheduled Job (BURSUBM)panel as shown below:

  2. Insert parameter names into fields as required:
    Expiration Date/Time
    Day and time at which the submitter job will stop running. Setting the Expiration Date parameter to the value *TODAY causes the submitter job to expire automatically. If you leave the Expiration Date parameter blank, the submitter job does not expire.
    Maximum Number of Total Job
    Maximum number of jobs that can be submitted.

    At the end of the last of these submissions, the submitter job turns itself off even if the time to switch itself off has not been reached.

    Cycle Sleep Time
    Amount of sleep time (hhmmss: hours, minutes, and seconds) between two successive job submission cycles.
    Maximum Number of Jobs for Cycle
    Maximum number of jobs that can be submitted in a submission cycle. Assuming you want to submit one job per hour, set the Cycle Sleep Time parameter to 010000and theMaximum Number of Jobs for Cycleparameter to00001.
    Submit Already Scheduled Job
    An indicator of whether to submit the jobs that have already been scheduled before the submitter job. A forward slash (/) indicates that the option is selected. If this option is not selected, the jobs previously scheduled from the PC will be ignored.
    JES2 Input Class
    Input class in which the submitter job must be executed. The JES2 Input Class parameter updates the KURJBNPC job tabsheet located in the PDS SKEL product library. Before submitting the submitter job, you must customize this tabsheet according to your company standards.
    Enqueue Jobs with Same Name
    Indicates whether or not to enqueue a job if another job with the same name is already in the queue. When the value is a forward slash (/), only jobs with unique names are submitted and jobs with the same name as a job that is already in the queue are enqueued in a Data Express queue. When blank, all jobs are submitted without regard to the names of other jobs already running.
Parent topic: Process for z/OS Guide


GDG Support

The generation data group (GDG) data sets consist of one GDG base index file and generation data sets (GDSs) within a GDG data group. The generations within a GDG are assigned names derived from the name of the GDG base. Their names can be specified via relative name (signed integer) or absolute generation name.

Data Express loads GDG files into the knowledge base via Load Data Store Information from External Interface (job BURLFIL), passing the generations to the job using SEQ file with a special format.

Start with a GDG base file and three generations (GDSs) created on mainframe. For example:

XXXXXX.GDG.TEST
XXXXXX.GDG.TEST.G0001V00
XXXXXX.GDG.TEST.G0002V00
XXXXXX.GDG.TEST.G0003V00

You can specify the generation names in the SEQ file using either of the two following methods:

1. Relative Name (signed integer)
The SEQ file passed to the BURLFIL job must contain these names:
GDG  XXXXXX.GDG.TEST(0)
GDG  XXXXXX.GDG.TEST(-1)
GDG  XXXXXX.GDG.TEST(-2)
The three GDG generations are loaded as GDG file types in the Data Builder:
Data Store Name | Data Store Type
XXXXXX.GDG.TEST(000) | GDG
XXXXXX.GDG.TEST(-001) | GDG
XXXXXX.GDG.TEST(-002) | GDG
2. Absolute Generation Name
The SEQ file passed to the BURLFIL job must contain these names:
SEQ   XXXXXX.GDG.TEST.G0001V00
SEQ  XXXXXX.GDG.TEST.G0002V00
SEQ  XXXXXX.GDG.TEST.G0003V00
The three GDG generations are loaded as SEQ file types in the Data Builder:
Data Store Name | Data Store Type
XXXXXX.GDG.TEST.G0001V00 | SEQ
XXXXXX.GDG.TEST.G0002V00 | SEQ
XXXXXX.GDG.TEST.G0003V00 | SEQ
Parent topic: Process for z/OS Guide


z/OS Toolkit Guide

Outlines the concepts and procedures used by the Data Express Toolkit, which is a collection of utilities that help provide input to Data Express processes in order to automate data inventory functionality. It shows how to start and work with the toolkit in a mainframe environment.

This guide is for users who want to verify the integrity of data contained in databases (DB2 or DL/I) or data sets (VSAM, SEQUENTIAL, or GDG) of applications running on z/OS systems that are either developed internally or purchased by external suppliers.

The use of Data Express requires a minimum experience of z/OS.

  • Considerations
    Describes some considerations for this product.
  • Load Sequential File with DB2 Catalog
    Describes the load sequential file process.
  • Load IMPSRC for Life Cycle
    Describes the loading the IMPSRC file for Life Cycle.
  • DB2 Catalog Synchronization
    Describes the DB2 catalog synchronization.
  • Create CSV File from Referential Integrity
    Describes the create a CSV file from referential integrity process.
  • Load Sequential File with ADABAS FDT Report
    Describes the Load Sequential File with ADABAS FDT Report.
  • Toolkit Tables
    Describes the toolkit tables.
Parent topic: Data Express


Considerations

Bear in mind the following points about Data Express:

  • You do not need to analyze the sources of the programs in the application in order to use Data Express.
  • This product does not perform any operation capable of altering the analyzed data.
  • To use this product, you must have the correct licence. Refer to the Installation Guide for more details.

The tables defined during the installation phase and updated by the toolkit are described in the Appendix A. Toolkit Tables.

Parent topic: z/OS Toolkit Guide


Load Sequential File with DB2 Catalog

The process Load Sequential File reads the DB2 system catalog (starting from the list of tables that need to be loaded into the knowledge base), and then produces the sequential file containing the information needed for the load. After a new Load File From External Interface job is created and scheduled, the sequential file is used when the job executes to perform the load.

For more information about the data inventory process, see the section Data Inventory in the chapter Project Implementation in the Process Guide for z/OS.

For information about the structure of the output sequential file, see the section Direct Access - API Load Sequential File in the chapter Sequential Files in the Data Model Guide.

Note: The filename listed in the FILENAME field can be indicated by a partial name or the special value '*ALL'. The function will then populate all the DB2 tables with the name in accordance with the value specified.

To load the sequential file with the DB2 catalog:

  1. From the Main Menu panel, enter the following command: TSO CUINFIL.
  2. Specify the company name and press Enter.
  3. Insert parameter names into all fields as required:
    Data Set Name Inp

    Name of the sequential file which contains the list of the files that must be loaded into Data Express.

    Data Set Name Out

    Name of the sequential file which will be created from this function, with all the information about the files that must be loaded into Data Express.

    Process Id

    Identifier for the process.

    Select Count

    Type of select to get the total number of records:

    1. select count on DB2 table (slowest).
    2. select on SYSIBM.SYSTBALES (fastest).
    3. select on SYSIBM.SYSTBALES and if there aren't statistics, a select count on the DB2 table is performed.
Parent topic: z/OS Toolkit Guide


Load IMPSRC for Life Cycle

The Life Cycle is a feature that enables automatic updates for file layout information. The Life Cycle process can use a guide file, which is a sequential file with a name that is similar to the project and group specification and followed by the IMPSRC file. It can be completed by the user (using an interface from Endevor, Changeman, or any other change management tool) or directly with the job Load Sequential for Life Cycle.

The job Load Sequential for Life Cycle writes the IMPSRC file (starting from a PDS with the copy loaded in the knowledge base or from a DB2 owner (schema)) with all the copy or the new or changed DB2 tables. After a new job is created and scheduled, the IMPSRC file is used when the job executes to schedule the Life Cycle phases.

Note: The Life Cycle process uses only one IMPSRC file. Therefore, we recommend that the process be executed for every DB2 owner (database schema) or for every partitioned data set.

For more information about the Life Cycle, see the chapter Life Cycle in the Process Guide for z/OS.

To load the IMPSRC file for the Life Cycle:

  1. From the Main Menu panel, enter the following command: TSO CUINLIC.
  2. Insert parameter names into all fields as required:
    Data Set Type

    Mode with which you want to work.

    Note: Like the Analyze FDT report, the Life Cycle for ADABAS lets you analyze an FDT report, but with the difference that you can analyze multiple FDT Reports simultaneously.
    Data Set Name

    Name of the partitioned data set or DB2 Owner (schema name) with which you want to work.

    Note: If you are using Life Cycle for ADABAS, the data set name must be the name of the PDS file that contains the desired FDT Report as a member.
    Select Count

    Type of select to get the total number of records:

    1. select count on DB2 table (slowest).
    2. select on SYSIBM.SYSTBALES (fastest).
    3. select on SYSIBM.SYSTBALES and if there are no statistics, a select count on the DB2 table is performed.
Parent topic: z/OS Toolkit Guide


DB2 Catalog Synchronization

Use the z/OS process DB2 Catalog Synchronization to produce the sequential file containing the information needed for the load, unless the file can be produced in another manner. After a new DB2 Catalog Synchronization (client) job is created and scheduled, the sequential file is used when the job executes to load information for the cardinality for DB2 tables and the existence of a key or the existence of an image copy into the knowledge base.

The DB2 Catalog Synchronization (client) job should be executed before beginning the data subset extraction process, so that all the information is synchronized and this process can be independent of the system tables. For more information about the job, see the section DB2 Catalog Synchronization Job in the chapter Work with Jobs in the Front End Guide.

Note: For information about the structure of the output sequential files for HEADER, INTERNAL KEY, EXTERNAL KEY, and IMAGE COPY see the section DB2 Catalog Synchronization Information in the chapter Sequential Files in the Data Model Guide.

To synchronize the DB2 catalog:

  1. From the Main Menu panel, enter the following command: TSO CUINSYNC.
  2. Indicate the scope of the process by specifying the DB2 tables you need in the DB2 Owner and Table Name fields, and the name of the sequential file in the Data Set Name field.
  3. Set the Select Count parameter to retrieve the total number of records in a DB2 table, using one of these options:
    • Select count on DB2 table – the slowest option, default setting
    • Select count on SYSIBM.SYSTABLES – the fastest option
    Note: For the DB2 Owner and Table Name fields, you can specify *ALL if you want to process all owner names or all table names.
Parent topic: z/OS Toolkit Guide


Create CSV File from Referential Integrity

The process Create CSV file from Referential Integrity analyzes a SQL DDL generated by the Erwin Data Modeler that describes the referential integrity relation using the IBM DB2 standard, and then creates a CSV file that contains all relations described in the DDL. After a new Import Method From Referential Integrity job is created and scheduled, the CSV file is used when the job executes to create the method needed to generate the test environment. For more information about method creation, see the chapter Data Subsetting in the Process Guide for z/OS.

To create the CSV file from referential integrity:

  1. From the Main Menu panel, enter the following command: TSO CVTEARI.
  2. Insert parameter names into fields as required:
    PDS or Data set name
    Name of the DDL source data set or PDS containing the referential integrity relation generated by the Erwin Data Modeler.
    Source Name
    Name of the member of PDS that contains the DDL generated by the Erwin Data Modeler.
    CSV data set name
    Name of the CSV data set that will contain the referential integrity relation. The CSV data set will be created if it does not exist. If it already exists, it will be written in append mode.
Parent topic: z/OS Toolkit Guide


Load Sequential File with ADABAS FDT Report

The process Analyze ADABAS FDT report reads the FDT Report source sequential file (generated by the command ADAREP REPORT) and then produces the output sequential file containing the information needed for the load. After a new Load Data Store From External Interface job is created and scheduled, the output sequential file is used when the job executes to perform the load.

For more information about the Data Inventory process, see the section Data Inventory in the chapter Project Implementation in the Process Guide for z/OS.

For information about the structure of the output sequential file, see the section ADABAS - API Load Sequential File in the chapter Sequential Files in the Data Model Guide.

To load the sequential file with the ADABAS FDT report:

  1. From the Main Menu panel, enter the following command: TSO CVURAAD.
  2. Select the appropriate Machine ID and company name, and press Enter. The Analyze ADABAS FDT report (BURAAD) panel displays.
  3. Insert parameter names into fields as required:
    PDS or data set name
    Name of the PDS or sequential file that contains the FDT report ADABAS files.
    Source Name
    Name of the member that contains the FDT report ADABAS files.
    Data Set Name Out
    Name of the sequential file to create which will contain all the information about the ADABAS files that must be loaded into Data Express.
Parent topic: z/OS Toolkit Guide


Toolkit Tables

During toolkit installation, the following tables are created:

Table Name | Description
HSSYSCKS | View of SYSIBM.SYSCHECKS
HSSYSCOL | View of SYSIBM.SYSCOLUMNS
HSSYSDAU | View of SYSIBM.SYSDBAUTH
HSSYSDBA | View of SYSIBM.SYSDATABASE
HSSYSDEP | View of SYSIBM.SYSCHECKDEP
HSSYSFKE | View of SYSIBM.SYSFOREIGNKEYS
HSSYSIND | View of SYSIBM.SYSINDEXES
HSSYSIPA | View of SYSIBM.SYSINDEXPART
HSSYSKEY | View of SYSIBM.SYSKEYS
HSSYSRAU | View of SYSIBM.SYSRESAUTH
HSSYSREL | View of SYSIBM.SYSRELS
HSSYSSYN | View of SYSIBM.SYSSYNONYMS
HSSYSTAB | View of SYSIBM.SYSTABLES
HSSYSTAU | View of SYSIBM.SYSTABAUTH
HSSYSTPA | View of SYSIBM.SYSTABLEPART
HSSYSTSP | View of SYSIBM.SYSTABLESPACE
HSSYSVDE | View of SYSIBM.SYSVIEWDEP
HSSYSVIE | View of SYSIBM.SYSVIEWS
HSURDCOL | View of SYSIBM.SYSCOLUMNS
HSURDSYC | View of SYSIBM.SYSCOPY
Parent topic: z/OS Toolkit Guide


Data Generation Guide

This guide outlines the concepts and procedures used by the Data Express Data Generation module, which uses user-specified selection criteria, rules, and data to create a new test environment from scratch.

You begin with an existing but empty test environment. Then, use the Data Generation module to:

  • Create rules that derive a test environment from the client side
  • Populate job submissions derived from the client side
  • Load tables via the Distributed Loader for distributed data stores, or via the external interface for z/OS data stores
  • Define and associate selection classes

Use this process to generate a full test environment, or a partial test environment. A partial test environment typically includes the applications not present in production. In this case, the remainder of the test environment comes from production through Data Subset Extraction.

  • Who Should Read This Guide
    Describes the audience for the Data Generation guide.
  • Before Starting with Data Generation
    Describes prerequisites for using the Data Generation module.
  • Getting Started
    Provides an overview of the Data Generation module. We recommend that you start by reading this section before using the module.
  • Directive Types
  • Distributed Exporter Utility
  • Executing Data Generation
Parent topic: Data Express


Who Should Read This Guide

This guide is for Micro Focus users who are interested in using a subset of data to populate empty tables independent of the content of a production environment. It explains the Data Generation configuration procedure, and describes the steps required to populate the files in your application.

Before using this product, we recommend that you carefully review the other Data Express documentation, including the Front End Guide and the Data Subset Extraction Guide in particular.

Parent topic: Data Generation Guide


Before Starting with Data Generation

Before using the Data Generation module for Data Express bear in mind the following points:

  • The Data Generation module is available for both Data Express for Distributed Systems and Data Express for z/OS. This requires that you have the correct license. See the Installation Guide for details.
  • To use the Data Generation module, it is not necessary for you to analyze application or program source code.
  • The Data Generation module manages all types of z/OS data stores processed by the Data Builder module except DL/I. , in addition to all types of ODBC distributed data stores processed by the Data Builder module.
Parent topic: Data Generation Guide


Getting Started

Provides an overview of the Data Generation module. We recommend that you start by reading this section before using the module.
  • Mode
  • Data Generation UI Guidelines
  • Starting the Data Generation Module
Parent topic: Data Generation Guide


Mode

Data Express provides two modes:

Standard Mode
Supports direct access to the data resident on RDBMS from your PC.
Client/Server Mode

Supports access to the data resident on RDBMS through the Data Express kbde-Server, which is based on a three-tier architecture with the second tier handling data transfer optimization. This enables you to access data over the Internet with a minimal amount of administration and using efficient network capacity. In addition, Client/Server mode can use the same functions as Standard mode, but does not require any additional software such as the DB2 client or the Borland Database Engine (BDE).

Data Express for z/OS can be used in either Standard or Client/Server mode, whereas Data Express for Distributed Systems can be used in Standard mode only.

For Data Subset Extraction configuration information and instructions, see the Appendix A. Standard Mode and Appendix B. Client/Server Mode sections in the Front End Guide.

Parent topic: Getting Started


Data Generation UI Guidelines

The following guidelines apply to windows and screen components of the Data Generation module:

  • The main window, Data Generation - [Master Form], cannot be closed except when exiting the module.
  • Multiple MDI windows may be open at the same time.
  • To view a list of commands that apply to a specific item or area, right-click on the item or area.
Parent topic: Getting Started


Starting the Data Generation Module

To start the Data Generation module:

  1. Click Start > All Programs > Micro Focus Data Express 4.0 > Data Generation.
    Note: If you are using Data Express on Windows Vista and User Access Control is enabled, you must run Data Generation as an Administrator.
  2. Select the required database with which to launch the connection. By default, the highlighted item in the list of available databases is the last database you connected to using the Data Generation module.
    Note: To connect to the database for first time, you need your user ID and password.
  3. Select the required schema. The Data Generation - [Master Form] window appears:
    Figure 1. Data Generation - [Master Form] Window

    Data Generation - Main Window

Tip: Once you are connected to a database, you can open and close a database from the Data Generation - [Master Form] window by clicking File > Open or File > Close.
Parent topic: Getting Started


Directive Types

The following directives, applied at column level, enable the generation of synthetic data according to their defined rules. If no directive is specified for a given column, the column is populated with the default value for column type; for example, zero for numeric columns:

Generation by dictionary
Column values are obtained from a dictionary. Data Express provides some pre-loaded dictionaries (for masking purposes), and allows you to add dictionaries as well. To generate using a dictionary, you specify the dictionary name in the directive properties.
Generation by domain value list
Column values are obtained from a user-defined list. For example,if your column is a CHAR(1) and you want to assign it the values M and F (as male and female) with probability distribution values to represent 60% M and 40% F, you can specify these values in the directive properties.
Generation by random value list
Columns are randomly assigned values in the column's domain of definition.
Generation by referential integrity (other file dependency)
Column values are obtained from a list of values present in a column from another table. To ensure this, the parent and child column must be of the same class.
Note: This feature is similar to the "filter by filtered list" feature in Data Subset Extraction.
Parent topic: Data Generation Guide


Distributed Exporter Utility

The Distributed Exporter utility that creates and provides necessary information to the Extension Technology that enables the generation of a distributed data store.

This utility generates the following files after you have successfully exported from the Data Generation module:

Draft comment:
Can you export from the Data Generation module? I don't see anything in the document that explains the process.
method.rc
Coded content of source and target databases information.
method.txt
Information for the environment and the method in it which are about to be masked.
filter.txt
All created filters in the test environment.
elab.txt
Information about tables involved in Data Generation process.
Dir.txt
Information about the directives to be used
cbfld.txt
Information about combined fields used in the method.
CREATETABLE.sql
A list of the tables that need to be masked; a generic CREATE TABLE statement is provided for each table.
CREATEINDEX.sql
A list of all the indexes for the tables that need to be masked; a generic CREATE INDEX statement is provided for each index.
ALTERTABLE_RI.sql
A list of all the primary and foreign keys for the tables that need to be masked; a generic ALTER TABLE statement is provided for appropriate primary and foreign keys.

There are two areas of interest when it comes to using the Distributed Exporter with distributed data stores.

  • If you use ODBC-enabled data stores, you can subset across all your data stores with one invocation of the ODBC Extension. This feature is not available with the Oracle Extension.

  • You also have the ability to subset within one distributed data store, as long as a distinct target schema name is provided.

If you use the Distributed Exporter utility on one machine, and do your generation on another, make sure that the target ODBC DSN matches the ODBC DSN used for the actual data generation.

Parent topic: Data Generation Guide


Executing Data Generation

Data Generation execution for distributed data stores is done from the command line using the dxestart command, which reads the configuration files exported by Distributed Exporter, and populates the target environment according to the laws and directives specified.

The dxestart executable resides in the C:\Program Files (x86)\Micro Focus\Data Express 4.0\Synthetic\ODBC directory by default.

For complete information about using the dxestart command, see The dxestart Command topic in your Getting Started with Distributed Data Stores documentation.

  • Catalog Distributed Data
  • Catalog z/OS Data
  • Classify Data
  • Define Rules
  • Export Rules
  • Execute Rules
  • Data Generation Jobs
Parent topic: Data Generation Guide


Catalog Distributed Data

You catalog the target database in Data Express using the Distributed Loader feature of the Data Builder module. A detailed description of this feature is in the Using Distributed Loader section of your Getting Started with Distributed Data Stores documentation. Consider the following:

  • When using Distributed Loader, the target database should be accessible via ODBC using the dxegenloadfile utility either from the machine where the Data Builder module is installed or from the machine where the distributed engine is installed. The dxegenloadfile utility is detailed in your Getting Started with Distributed Data Stores documentation.
  • Distributed Loader allows you to specify an ODBC instance and schema, and presents a list of tables from which you select tables to load into the Data Express knowledge base.
  • After confirming which tables to load, Distributed Loader launches an interactive process that writes the catalog and structure information for the selected tables to the knowledge base.
Parent topic: Executing Data Generation


Catalog z/OS Data

For details on the process of cataloging z/OS data in Data Express, see the Getting Started with z/OS Data Stores topics.

Parent topic: Executing Data Generation
Related concepts
Getting Started with z/OS Data Stores


Classify Data

The Data Generation module requires that you assign a class to each column that uses referential integrity directives. The process used to classify distributed and z/OS data stores is essentially the same. For specific instructions on assigning classes for distributed data stores, see the Importing classes section in your Getting Started with Distributed Data Stores documentation. For instructions on assigning classes for z/OS Data, see Getting Started with z/OS Data Stores.

Note: While you can assign classes to columns that are not used in Data Generation, it is not required.
Parent topic: Executing Data Generation


Define Rules

Use the Create Code wizard to define Data Generation rules for both distributed and z/OS data stores. The Create Code page of the wizard enables you to create, name, and describe a code set for a given machine and company:

Create Code page

After applying the parameters specified on the Create Code page, you then progress to the Work With Directives page. On this page, you associate the code set with tables by adding a law set rule for each table. Each law rule definition includes a parameter that specifies the number of records used to populate the associated table.

Work With Directives

From this page, you can open each law rule, enabling you to specify directives when necessary. To do this, right-click a Column Name; then select Set Directive. This opens the Create a Directive for column page.

You can select the directive type, and assign a routine name to the directive type. Each directive type obtains column values differently as follows:

Other file dependency
These directives obtain all relevant information from the class assignment. The values for the new column are obtained from the column in the parent table that is associated with the same class.

Create a Directive for column, Other file dependency

Create a Directive for column page

Dictionary
These directives require that you specify the dictionary name. For further information on Data Express dictionaries, see the Masking Routines section in your Getting Started with Distributed Data Stores documentation.

Create a Directive for column, Dictionary

Create a Directive for column page Dictionary

Domain value list
These directives require that you specify the different values the column can have, and a frequency for each value.
Important: Columns with no directives associated receive the default value for the associated column type.
Parent topic: Executing Data Generation


Export Rules

For distributed data stores only, you can use the Export wizard to export rules at the code set level. The wizard enables you to choose the machine, company, and code, and to specify the output directory. The output directory should be the folder that contains the configuration files.

Note: The List of columns and enabled directives is on the Export page as a summary, and is not editable.
Parent topic: Executing Data Generation


Execute Rules

For distributed data stores only, you can execute Data Generation rules. However, to do this you must have a run-time knowledge base. A run-time knowledge base is tables in the same database instance that contains the target tables to be populated, or in a separate instance if you use a centralized run-time knowledge base. Details about run-time knowledge base creation, including a centralized run-time knowledge base, are described in the DBA Tasks for Extension Technology section of your Getting Started with Distributed Platforms documentation.

Once the run-time knowledge base is in place, the execution of rules is done by executing the dxestart command. See The dxestart Command topic in your Getting Started with Distributed Platforms documentation.

Parent topic: Executing Data Generation


Data Generation Jobs

For Data Express for z/OS only, the Work with Jobs area contains information about the function related to the execution of jobs for test environment generation and is part of the Data Builder module. Jobs are submitted using the Work with Jobs area of Data Builder.

For more information about the Work with Jobs area, see Work with Jobs in the Front End Guide.

  • Introduction to Data Generation Jobs
  • Job Creation
Parent topic: Executing Data Generation


Introduction to Data Generation Jobs

To access the Work with Jobs area:

  1. Start Data Builder by clicking Start > All Programs > Micro Focus Data Express 4.0 > Data Builder.
  2. Click [image] (Work with Jobs) in the toolbar of the Data Builder main window.
Parent topic: Data Generation Jobs


Job Creation

This section describes the procedures for creating jobs relevant to data generation.

  • Data Generation Jobs
  • Create Allocation Jobs
  • Create Reload Jobs
Parent topic: Data Generation Jobs


Data Generation Jobs

After all files have been allocated for extraction, launching the Data Generation job is the final step in creating the test environment.

To create a Data Generation job:

  1. Create a new Data Generation job from the Work with Jobs area in the Data Builder module. See Job Creation in the Front End Guide for basic instructions. The Secondary Options window for the Data Generation job is displayed.
  2. Specify the name of the Code from the corresponding drop-down list. If desired, specify the range of steps for which you want to launch the elaboration.
  3. Click OK to submit the job.
  4. Wait for the data generation phase to be completed.
Parent topic: Job Creation


Create Allocation Jobs

The Create Allocation job executes a process to create a JCL that will delete and allocate all the output for the data set associated with the data Generation according to a given code.

  • Job Creation
  • JCL Information
Parent topic: Job Creation


Job Creation

To create a Create Allocation job for Data Generation:

  1. From the Work with Jobs window, click New.
  2. In the List of Jobs section, select Create Allocation Jobs for Data Generation.
  3. In the General section, select the appropriate Machine Id and Company name.
  4. Click Apply to submit the job. The Secondary Options window appears:
    Secondary Options - Data Generation
  5. In the Secondary Options window, select the required Code value, and specify values for the following fields:
    From Step / To Step
    Range of values for a subset of steps for the selected code.
    Max no. of files in JCL
    Maximum number of files you want process in a single JCL. If your code exceeds this limit, a new JCL is created for the other files. The default value of the parameter is 2000.
    Max cylinder no. for primary quantity
    Maximum number of cylinders that can be assigned to a primary quantity. The value of 0 indicates to use the default value of 4300 cylinders.
    Maximum cylinder No. for secondary quantity
    Maximum number of cylinders that can be assigned to a secondary quantity. The value of 0 indicates to use the default value of 4300 cylinders.
    Maximum size for disk data sets
    Maximum number of cylinders (not for the data set) to be allocated on an alternative unit. If the primary quantity exceeds this dimension, the data set will be allocated on an alternative unit.
    Alternative name for unit
    Name of the alternative unit where you want to allocate the data set that exceeded the Maximum size for disk data sets value.
    Primary quantity percentage
    Percentage of cylinder you want to assign at the primary quantity in regard to the total dimension of output data set. The default value is 10%.
    Secondary quantity percentage
    Percentage of cylinder you want to assign at the secondary quantity in regard to the total dimension of primary quantity. The default value is 15%.
    JCL output data set
    Library where the generated JCLs will be stored.
    JCL output member for deletion
    First part of the name of the generated JCLs (the second part will be a 3-digit progressive number starting from 001).
    JCL output member for allocation
    First part of the name of the generated JCLs (the second part will be a 3-digit progressive number starting from 001).
    Submit Job
    Indicator of whether to submit the newly created JCLs.
Parent topic: Create Allocation Jobs


JCL Information

The JCLs are built starting from the CBDGALS1 and CBDGALS2 skeletons, which can be customized.

The structure of the skeletons comprises a starting part and an ending part that are repeated once in the created JCLs, and a central part that is delimited by the following keywords:

//* SINGLE FILE ELABORATION and

//* SINGLE FILE END ELABORATION, that is repeated for each file.

In the skeleton, some variables are used and are then changed into the JCL values using the values stored in the Knowledge Base or included in the launch parameters.

The following table describes the used variables:

Variable | Description
&TYPE | Name of the sequential data set (output of extraction process)
&PCYL | Primary quantity
&LRECL | Record length
&UNIT | SYSDA or the value put in the launch screen, depending on the above-described rules
&CYL | TRK if the space is less than a cylinder, or CYL otherwise
&SCYL | Secondary quantity
&JOBN | It is actualized with the JCL name
Parent topic: Create Allocation Jobs


Create Reload Jobs

This function allows you to create a JCL to reload all DB2 tables with direct access included in a code.
  • Job Creation
  • JCL Information
Parent topic: Job Creation


Job Creation

To create a Create Reload job:

  1. From the Work with Jobs window, click New.
  2. In the List of Jobs section, select Create Reload Jobs for Data Generation.
  3. In the General section, select the appropriate Machine ID and Company name.
  4. Click Apply to submit the job. The Secondary Options window appears.
    Secondary Options - Data Generation
  5. In the Secondary Options window, select the required Code value, and specify values for the following fields:
    From Step / To Step
    Range of values for a subset of steps for the selected code.
    Max no. Data Stores in JCL
    Maximum number of data stores you want process in a single JCL. If your method exceeds this limit, the function creates a new JCL for the other files. The default value is 2000.
    Replace Data
    Indicator of whether data replacement in the target database is enabled. If not selected, data items are added without deletion.
    Create Syspunch
    Indicator of whether to create the part of the JCL performing the SYSPUNCH.
    Create Reload
    Indicator of whether to create the part of the JCL performing the reload.
    Target DB2 Owner
    Name of the target DB2 owner. If this parameter is not specified, this value is the same as cataloged in Data Express.
    Target DSN
    Name of the target database subsystem. If this parameter is not specified, this value is the same as cataloged in Data Express.
    JCL output data set
    Library where the generated JCLs will be stored.
    JCL output member for reload
    Name of the generated JCLs used for reload.
    JCL output member for repair
    Name of the generated JCLs used for repair.
    Submit Job
    Indicator of whether to submit the newly created JCLs.
Parent topic: Create Reload Jobs


JCL Information

The JCLs are built starting from CBDGLOD1 and CBDGLOD2 skeletons, which can be customized. The CBDUNLD2 skeleton is an alternative skeleton where the UNLOAD utility is used instead of the DSNTIAUL utility in order to create SYSPUNCHs.

The structure of the skeleton comprises a starting and ending part that are repeated once in the created JCLs, and a central part that is delimited by the following keywords:

//* SINGLE FILE ELABORATION and

//* SINGLE FILE END ELABORATION, that is repeated for each file.

In the skeleton some variables are used, and they are changed into the JCL values using the values stored in the Knowledge Base or included in the launch parameters.

The following table describes the used variables:

Variable | Description
&NUM | Numeric progressive
&UNLOUTNAM | Name of the sequential data set (output of extraction process)
&OWNAME | Owner of the original table
&FILNAME | Name of the original table
&DBNAME | Database name of the original table
&TSNAME | Tablespace name of the original table
&UNLINPNAM | Name of the unload as specified in the Work with files area
&STEP | Step name for the SYSPUNCH generation (automatically generated)
&STP1 | Step name for the MISYSPUN program call (automatically generated)
&STP2 | Name of the reload step (automatically generated)
&SYSPARM | It can be ADD,YES, number of sortkeys if no replace parameter, REP,YES, number of sortkeys if replace parameter
&DSNNAME | Name of the input DSN
&METHOD | Name of the code
&JOBN | Name of the job actualized with the JCL name
Parent topic: Create Reload Jobs


Glossary

A

Application Program Interface (API)
Standard interface for Data Express: it is a way to feed the knowledge base with external information (for example, change management, database design, and data base catalog). Data Builder needs a standard interface, and the API’s task is to fill this interface. APIs can be developed by Micro Focus or customers, but they are not part of Data Express.

C

client
Tool or suite of tools that provide a front-end interface to the knowledge base.
cataloging
Group of batch jobs populating the knowledge base with the technical structure information of the data stores (for example, name, size, data elements, and data element properties).
class
Abstract property of a field that describes its meaning, for example, Name, Amount, Account, or code.
classification
Group of data entry functions and batch jobs populating the knowledge base with the application information of the data stores (class-data elements association).
confidence
An abstract measurement that indicates how well the data types in the analyzed data stores match the data types in a copybook.

E

engine
Group of batch jobs creating the target environment data by applying the rules defined in the Knowledge Base to the files cataloged in the Knowledge Base. The output of the engine constitutes a flat file.
environment
Set of the files to be processed and moved into another place.
environment population
Group of data entry functions and batch job loading the engine result in the target environment.

F

fingerprint
A data element content analysis function that provides statistical information about the content of a data element.

K

Knowledge Base
Collection of tables within a database that contain all information about cataloged data stores, including properties and processing rules.

L

Life Cycle
Group of data entry functions and batch job keeping the knowledge base up to date when a change in the technical structure occurs, keeping intact the classification and rule definition information as much as possible.

M

method
Set of rules needed to process an environment.

P

Programmable Rules Definitions (PRD)
Code that can be used in order to add flexibility to Data Express. In addition to the standard in reduction and in masking rules, it is possible to call external code with a standard program interface (provided by Micro Focus) from the engine. PRDs can be developed by Micro Focus by or customers, but in any case they are not part of Data Express.

R

rules definition
Group of data entry functions and batch job populating the knowledge base with the masking and reduction rules to be applied to an environment.

S

step
Phase, marked by a successive number, which comprises a method. Operations belonging to the same step can be performed in an arbitrary order, whereas the performance of the steps must be performed in order according to the associated number.
Super class
A set of several different classes. During the class automatic assignment phase, this type of classification enables you to restrict the search
Parent topic: Data Express


Copyright

Copyright © 2003-2015 Micro Focus. All rights reserved.

Micro Focus has made every effort to ensure that this book is correct and accurate, but reserves the right to make changes without notice at its sole discretion at any time. The software described in this document is supplied under a license and may be used or copied only in accordance with the terms of such license, and in particular any warranty of fitness of Micro Focus software products for any particular purpose is expressly excluded and in no event will Micro Focus be liable for any consequential loss.

MICRO FOCUS, the Micro Focus logo, and Micro Focus product names are trademarks or registered trademarks of Micro Focus IP Development Limited or its subsidiaries or affiliated companies in the United States, United Kingdom, and other countries. All other marks are the property of their respective owners.

IBM® and CICS® are registered trademarks, and IMS™, DB2, DB2/390, DB2 Connect Enterprise Edition, DB2 Connect Personal Edition, COBOL for OS/390, Enterprise Systems Architecture/390, ESA/390, MVS, MVS/ESA, OS/390, S/390, System/390, VSE/ESA, and MFS are trademarks, of International Business Machines Corporation.

Netscape Enterprise Server™ is a trademark of Netscape Communications Corporation.

Internet Information Server®, SQL Server®, Visual Studio®, Windows 95®, Windows 98®, Windows NT 4.0®, Windows ME®, Windows 2000®, Windows Server®, Windows XP®, and Windows Vista® are registered trademarks of Microsoft Corporation in the United States and other countries.

Pentium ® is a registered trademark of INTEL Corporation.

Borland Database Engine™ and MIDAS™ are trademarks of Borland Software Corporation.

Oracle ™ is a trademark of Oracle Corporation Inc.

UNIX ® is a registered trademark of The Open Group.

Solaris™, Ultra Sparc™, and Java™ are trademarks of Sun Microsystems, Inc.

Linux, GCC compiler, GNU CTAGS, GNU DIFF, Samba, and KDE/GNOME refer to Open Source tools.

This product includes modified versions of software developed by the Apache Software Foundation (as described on the Apache web site).

The YGrep Search Engine is Copyright (c) 1992-2004 Yves Roumazeilles. All other trademarks are the property of their respective owners.

No part of this publication, with the exception of the software product user documentation contained on a CD-ROM, may be copied, photocopied, reproduced, transmitted, transcribed, or reduced to any electronic medium or machine-readable form without prior written consent of Micro Focus (IP) Ltd. Contact your Micro Focus representative if you require access to the modified Apache Software Foundation source files.

Licensees may duplicate the software product user documentation contained on a CD-ROM, but only to the extent necessary to support the users authorized access to the software under the license agreement. Any reproduction of the documentation, regardless of whether the documentation is reproduced in whole or in part, must be accompanied by this copyright statement in its entirety, without modification.

U.S. GOVERNMENT RESTRICTED RIGHTS. It is acknowledged that the Software and the Documentation were developed at private expense, that no part is in the public domain, and that the Software and Documentation are Commercial Computer Software provided with RESTRICTED RIGHTS under Federal Acquisition Regulations and agency supplements to them. Use, duplication or disclosure by the U.S. Government is subject to restrictions as set forth in subparagraph (c)(1)(ii) of The Rights in Technical Data and Computer Software clause at DFAR 252.227-7013 et. seq. or subparagraphs (c)(1) and (2) of the Commercial Computer Software Restricted Rights at FAR 52.227-19, as applicable. Contractor is Micro Focus (IP) Ltd,

9420 Key West Avenue, Rockville, Maryland 20850 . Rights are reserved under copyright laws of the United States with respect to unpublished portions of the Software.

2018-04-12

Parent topic: Data Express