August 2005 Technical Tip - Variable Length Records: VB, RDW, BDW, Oh my!

August 2005 Technical Tip - Variable Length Records: VB, RDW, BDW, Oh my!

One-to-many relationships are common in business systems. For example, one sales rep can have many customers, one customer can have many invoices, one invoice can have many items, etc. In modern systems, these relationships would likely be implemented with a relational database system using joins of multiple tables. In legacy systems, such relationships were commonly implemented in a hierarchical database or, more simply, with variable length records. This article will focus on how variable length records are stored internally.

To illustrate, consider the following credit card example. Let's assume one person can have many credit cards, but each credit card belongs to one and only one person. Since each record will have a variable number of credit cards, the record will typically include a field which indicates how many. We will choose an arbitrary maximum of 3 credit cards per person (artifically low to simplify our example.) The fixed portion of each record will contain an ID number (3 bytes), first name (10 bytes), last name (10 bytes), and occurances count (2 bytes), for a total of 25 bytes. For each credit card we will store the credit card number (4 bytes), credit card type (4 bytes) and balance due (4 bytes), for a total of 12 bytes. So each record contains from 25 bytes to 61 bytes (25 + (3*12)).

When the system stores a variable length record, it puts a four byte record descriptor word (RDW) on the front of each record. This RDW is not always visible (such as within a COBOL program) but it's there. The first two bytes of the RDW is a binary halfword (PIC S9(4) COMP) containing the length of the record. This length includes the data and the RDW itself. Consequently, our RDW will have a value of 29 to 65. This highest number (65) will be the LRECL when we allocate the file. The second two bytes of the RDW contains X'0000'.

We can allocate a simple variable length file with ISPF panel 3.2. For example:

   Menu  RefList  Utilities  Help                                               
 ______________________________________________________________________________ 
                             Allocate New Data Set                              
 Command ===>                                                                   
  
 Data Set Name  . . . : U0290.CREDIT.CARDS                                      
                                                                                
 Management class . . . MCTSOX         (Blank for default management class)     
 Storage class  . . . . SCBASE         (Blank for default storage class)        
  Volume serial . . . . XTSO05         (Blank for system default volume) **     
  Device type . . . . .                (Generic unit or device address) **      
 Data class . . . . . .                (Blank for default data class)           
  Space units . . . . . TRACK          (BLKS, TRKS, CYLS, KB, MB, BYTES         
                                        or RECORDS)                             
  Average record unit                  (M, K, or U)                             
  Primary quantity  . . 1              (In above units)                         
  Secondary quantity    1              (In above units)                         
  Directory blocks  . . 0              (Zero for sequential data set) *         
  Record format . . . . VB                                                      
  Record length . . . . 65                                                      
  Block size  . . . . . 0                                                       
  Data set name type  :                (LIBRARY, HFS, PDS, or blank)  *         
                                       (YY/MM/DD, YYYY/MM/DD                    
  Expiration date . . .                 YY.DDD, YYYY.DDD in Julian form

We can then edit the file with ISPF panel 2. Be sure to choose the Preserve VB record length option!

   Menu  RefList  RefMode  Utilities  Workstation  Help                         
 ______________________________________________________________________________ 
                                Edit Entry Panel                                
 Command ===>                                                                   
                                                                                
 ISPF Library:                                                                  
    Project . . . U0290                                                         
    Group . . . . ISPF     . . .          . . .          . . .                  
    Type  . . . . CNTL                                                          
    Member  . . .                 (Blank or pattern for member selection list)  
                                                                                
 Other Partitioned, Sequential or VSAM Data Set:                                
    Data Set Name . . . credit.cards                                            
    Volume Serial . . .           (If not cataloged)                            
                                                                                
 Workstation File:                                                              
    File Name . . . . .                                                         
                                         Options                                
 Initial Macro  . . . .                     Confirm Cancel/Move/Replace         
 Profile Name . . . . .                     Mixed Mode                          
 Format Name  . . . . .                     Edit on Workstation                 
 Data Set Password  . .                  /  Preserve VB record length

We can enter the data as follows. Note that although the LRECL is 65, the ruler (from the COLS command) only goes out to column 61. The difference is the RDW (4 bytes) which is not seen.

   File  Edit  Edit_Settings  Menu  Utilities  Compilers  Test  Help            
 _______________________________________________________________________________ 
 EDIT       U0290.CREDIT.CARDS                              Columns 00001 00061 
 Command ===>                                                  Scroll ===> HALF 
 ****** ***************************** Top of Data ******************************
 =COLS> ----+----1----+----2----+----3----+----4----+----5----+----6-           
 000001 987BILL      QUALLS    035555VISA10002345MAST05001234DISC0100           
 000002 123RUBECCA   ALINIAZEE 011010VISA0000                                   
 000003 786ERIC      ALBERT    022323MAST01003434DISC9999                       
 000004 432JASON     ROGERS    017777DISC0750                                   
 ****** **************************** Bottom of Data ****************************

IDCAMS can be used to dump the file. The JCL appears as follow:

   File  Edit  Edit_Settings  Menu  Utilities  Compilers  Test  Help            
 _______________________________________________________________________________ 
 EDIT       U0290.ISPF.CNTL(DUMPVB) - 01.00                 Columns 00001 00072 
 Command ===>                                                  Scroll ===> HALF 
 ****** ***************************** Top of Data ******************************
 000001 //U0290DMP JOB (PRM,0,T),'BILL QUALLS',                                 
 000002 //  CLASS=A,MSGCLASS=T,TIME=(0,59),NOTIFY=U0290                         
 000003 //STEP010 EXEC PGM=IDCAMS                                               
 000004 //MYINPUT  DD DSN=U0290.CREDIT.CARDS,DISP=SHR                           
 000005 //SYSIN    DD *                                                         
 000006   PRINT INFILE(MYINPUT) DUMP                                            
 000007 //SYSPRINT DD SYSOUT=*                                                  
 ****** **************************** Bottom of Data ****************************

We can see from the following dump that the records are, in fact, varying in length.

LISTING OF DATA SET -U0290.CREDIT.CARDS RECORD SEQUENCE NUMBER - 1 000000 F9F8F7C2 C9D3D340 40404040 40D8E4C1 D3D3E240 404040F0 F3F5F5F5 F5E5C9E2 *987BILL QUALLS 035555VIS* 000020 C1F1F0F0 F0F2F3F4 F5D4C1E2 E3F0F5F0 F0F1F2F3 F4C4C9E2 C3F0F1F0 F0 *A10002345MAST05001234DISC0100 * RECORD SEQUENCE NUMBER - 2 000000 F1F2F3D9 E4C2C5C3 C3C14040 40C1D3C9 D5C9C1E9 C5C540F0 F1F1F0F1 F0E5C9E2 *123RUBECCA ALINIAZEE 011010VIS* 000020 C1F0F0F0 F0 *A0000 * RECORD SEQUENCE NUMBER - 3 000000 F7F8F6C5 D9C9C340 40404040 40C1D3C2 C5D9E340 404040F0 F2F2F3F2 F3D4C1E2 *786ERIC ALBERT 022323MAS* 000020 E3F0F1F0 F0F3F4F3 F4C4C9E2 C3F9F9F9 F9 *T01003434DISC9999 * RECORD SEQUENCE NUMBER - 4 000000 F4F3F2D1 C1E2D6D5 40404040 40D9D6C7 C5D9E240 404040F0 F1F7F7F7 F7C4C9E2 *432JASON ROGERS 017777DIS* 000020 C3F0F7F5 F0 *C0750 *

But what about the RDW? The RDW is usually not shown, but there is a trick you can use to force its display. This involves adding DCB=RECFM=U to the JCL as follows:

   File  Edit  Edit_Settings  Menu  Utilities  Compilers  Test  Help            
 _______________________________________________________________________________ 
 EDIT       U0290.ISPF.CNTL(DUMPVB) - 01.00                 Columns 00001 00072 
 Command ===>                                                  Scroll ===> HALF 
 ****** ***************************** Top of Data ******************************
 000001 //U0290DMP JOB (PRM,0,T),'BILL QUALLS',                                 
 000002 //  CLASS=A,MSGCLASS=T,TIME=(0,59),NOTIFY=U0290                         
 000003 //STEP010 EXEC PGM=IDCAMS                                               
 000004 //MYINPUT  DD DSN=U0290.CREDIT.CARDS,DISP=SHR,DCB=RECFM=U 
 000005 //SYSIN    DD *                                                         
 000006   PRINT INFILE(MYINPUT) DUMP                                            
 000007 //SYSPRINT DD SYSOUT=*                                                  
 ****** **************************** Bottom of Data ****************************

The resulting dump is at the same time confusing and revealing:

LISTING OF DATA SET -U0290.CREDIT.CARDS RECORD SEQUENCE NUMBER - 1 000000 00CC0000 00410000 F9F8F7C2 C9D3D340 40404040 40D8E4C1 D3D3E240 404040F0 *........987BILL QUALLS 0* 000020 F3F5F5F5 F5E5C9E2 C1F1F0F0 F0F2F3F4 F5D4C1E2 E3F0F5F0 F0F1F2F3 F4C4C9E2 *35555VISA10002345MAST05001234DIS* 000040 C3F0F1F0 F0002900 00F1F2F3 D9E4C2C5 C3C3C140 4040C1D3 C9D5C9C1 E9C5C540 *C0100....123RUBECCA ALINIAZEE * 000060 F0F1F1F0 F1F0E5C9 E2C1F0F0 F0F00035 0000F7F8 F6C5D9C9 C3404040 404040C1 *011010VISA0000....786ERIC A* 000080 D3C2C5D9 E3404040 40F0F2F2 F3F2F3D4 C1E2E3F0 F1F0F0F3 F4F3F4C4 C9E2C3F9 *LBERT 022323MAST01003434DISC9* 0000A0 F9F9F900 290000F4 F3F2D1C1 E2D6D540 40404040 D9D6C7C5 D9E24040 4040F0F1 *999....432JASON ROGERS 01* 0000C0 F7F7F7F7 C4C9E2C3 F0F7F5F0 *7777DISC0750 *

Look at the first line and you will see 00410000. This is the RDW for the first record. X'0041' is the hexadecimal representation of the number 65 (4 * 16 + 1). This is the total length of the first record. Find 00290000 in the third line. This is the RDW for the second record: 29 base 16 = 41 base 10. Similarly, you will see 00350000 in the fourth line and 00290000 in the sixth line, representing the lengths of the third (53) and fourth (41) records respectively.

Just as an RDW describes each record, a BDW describes each block. Find 00CC0000 in the first line. CC base 16 = 204 base 10. This is the length of the block: 65 (first) + 41 (second) + 53 (third) + 41 (fourth) + 4 (for the BDW itself.)

I hope this article helped you to understand variable length records. In next month's article I will show you how to reformat variable length records using SyncSort. Until then, we hope you will continue to consider Caliber Data Training when searching for an IT training provider.