<> ====================================================================== Microsoft(R) Product Support Services Application Note (Text File) WA0725: DIRECTORY SYNCHRONIZATION (DIR-SYNC) ====================================================================== Revision Date: 3/93 Disk Included The following information applies to Microsoft Mail for PC Networks version 3.0. -------------------------------------------------------------------- | INFORMATION PROVIDED IN THIS DOCUMENT AND ANY SOFTWARE THAT MAY | | ACCOMPANY THIS DOCUMENT (collectively referred to as an | | Application Note) IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY | | KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING BUT NOT LIMITED TO | | THE IMPLIED WARRANTIES OF MERCHANTABILITY AND/OR FITNESS FOR A | | PARTICULAR PURPOSE. The user assumes the entire risk as to the | | accuracy and the use of this Application Note. This Application | | Note may be copied and distributed subject to the following | | conditions: 1) All text must be copied without modification and | | all pages must be included; 2) If software is included, all files | | on the disk(s) must be copied without modification [the MS-DOS(R) | | utility DISKCOPY is appropriate for this purpose]; 3) All | | components of this Application Note must be distributed together; | | and 4) This Application Note may not be distributed for profit. | | | | Copyright 1993 Microsoft Corporation. All Rights Reserved. | | Microsoft and MS-DOS are registered trademarks of Microsoft | | Corporation. | -------------------------------------------------------------------- INTRODUCTION ============ This application note is designed for the mail administrator and outlines the procedures directory synchronization (Dir-Sync) performs during normal operation. The network administrator can use these procedures to examine Dir-Sync files, reset files, or perform a manual Dir-Sync process to fully synchronize the postoffice address lists. This application note is intended as a supplement to the "Microsoft Mail Administrator's Guide." GENERAL DESCRIPTION OF DIRECTORY SYNCHRONIZATION ================================================ Dir-Sync is a method of propagating local postoffice address list changes to other postoffices that are participating in the Dir-Sync process. Dir-Sync is meant to be invisible to both administrators and users; the changes are distributed automatically after the administrator has configured the Dir-Sync process. To accomplish synchronization, a directory server is defined. This directory server postoffice maintains the master list of all changes from requestor postoffices. When a change is made to a local address list, the change is sent to the Dir-Sync server, which incorporates the change into the master list and sends updates to the requestors. These updates only have the changes a postoffice needs to update its Global Address List (GAL). Complete the following nine steps to initialize directory synchronization: 1. Ensure that mail can be sent between the postoffices. 2. Turn on the GAL option. 3. Designate one postoffice as the Dir-Sync server. 4. Register all other postoffices as requestors with the server. 5. Set up the server's process time. 6. Register the server with all the other postoffices participating in Dir-Sync. 7. Set up the two times for the requestors' processes. 8. Export the user lists to the server. 9. Run the Dispatch program and run the External Mail program or a Message Transfer Agent (MTA). In more detail, these steps are: 1. Ensure that mail can be sent from each postoffice to all other postoffices that will be participating in Dir-Sync, by sending mail from each postoffice's Administrator account to every other postoffice's Administrator account. This step is required because Dir-Sync uses the mail system to transfer the address list updates, so regular mail must be able to be sent and received before Dir- Sync will function. 2. From the Administrator program, select Config, Global-List, Yes to enable the GAL. 3. Select Config, Dir-Sync, Options, Yes to make one postoffice the Dir-Sync server. 4. Select Config, Dir-Sync, Server, Requestors, Create to register each requestor postoffice on the server postoffice. 5. Select Config, Dir-Sync, Server, Schedule to schedule the time for the server to process the updates into the master list. 6. For each requestor, select Config, Dir-Sync, Requestor, Registration to designate the server postoffice. Specify the same postoffice that was designated in step 3 so that the requestor knows which postoffice to send the updates to. 7. On each requestor, run the Administrator program and select Config, Dir-Sync, Requestor, Schedule to set the Send Requests and Process Updates times. The requestor sends updates to the server at the specified Send Requests time, and it processes updates coming from the server at the Process Updates time. Make sure the requestor sends its requests to the server prior to the server's scheduled time to process requests, and make sure the requestor process updates after the server's scheduled time to send updates. Also, when setting the schedule for the requestor, add the additional time that the External Mail program or MTA needs to deliver the Dir- Sync mail. For example, the schedule might look like this: - Requestor's Send Requests time set to 10 P.M. - Server's Process Updates time set to 12 A.M. - Requestor's Process Updates time set to 2 A.M. In order for the Dir-Sync server to participate in Dir-Sync, it must be specified as a requestor as well. Registration is automatic once you designate the postoffice as the server, but the requestor's schedule must be set. If this is not done, the server will not send or receive directory updates. 8. Select Config, Dir-Sync, Requestor, Export to send the address list to the server postoffice. This will send the local address lists to the server. The server will build the master list and send out all the other postoffice lists to the other postoffices. This step is a "kick-start" to get the lists up to the server quickly. 9. To create the Dir-Sync mail and update the GAL automatically, the Dispatch program and External Mail program or MTA must be running and must be able to access each postoffice participating in Dir- Sync. The Dispatch program polls each postoffice. When the designated time for Dir-Sync to run has passed, the Dispatch program will create a mail message that has the directory changes as an attachment. The External Mail program or the MTA must be running between the requestor and the server postoffices to deliver the mail to the server and back to the requestor. Dispatch creates the update message, but the External Mail program or MTA must deliver it. You can run the Dispatch program and the External Mail program or MTA on the same PC by using the Idle (-I) option of Dispatch. By using -I and designating the External Mail program or MTA as the idle process, you can run mail delivery and directory exchange. The two executables share time on the same PC. TYPICAL DIR-SYNC SCENARIOS ========================== This section is a guide for setting up an efficient Dir-Sync configuration, involving the postoffice, the External Mail program or MTA, and the Dispatch program. In the following scenarios, assume that drive mappings start with the Dir-Sync server at drive M, and that the Dispatch program and the External Mail program or MTA are run from the same computer. A. STANDARD CONFIGURATION ========================= The following scenario has multiple requestors sending directories to the Dir-Sync server. The Dispatch program and the External Mail program or MTA can touch each postoffice. M N Server Requestor 1 \ / \ / Dispatch External / \ / \ O P Requestor 2 Requestor 3 The command line for the computer running Dispatch and External is: dispatch -dmp -i"external -dmp -a" If you use a gateway as the MTA, this command line will change. See the administrator's guide for the gateway you are using to determine the best approach for running the Dispatch program and the gateway MTA together. B. DYNAMIC DRIVE MAPPING ======================== In the following scenario, dynamic drive mappings connect Requestors 2 through 5 to the Dir-Sync server. Both the Dispatch and External Mail programs use the same dynamic drive table, so any postoffice the External Mail program delivers mail to should be included in Dir-Sync. M O Server Requestor 1 \ / \ / Dispatch Requestor 2 -------- External -------- Requestor 3 / \ / \ Requestor 4 Requestor 5 The command line for the computer running the Dispatch and External Mail programs is: dispatch -dmn -f -i"external -dmn -fop -a" Note that the Dispatch program does not need to designate which drives to use for the dynamic connections because it uses the first available drive designation, while the External Mail program does need this information. Thus, the Dispatch program requires the -F option, while the External Mail program requires -FOP to use drives O through P for dynamic connections. This configuration is not feasible with a gateway as the MTA because gateways as MTAs do not use dynamic drives. C. MODEM CONNECTIONS ==================== In the following scenario, modems are used to connect two networks together. Because the Dispatch program cannot run over the modem connection, two copies of Dispatch must be running simultaneously. Dispatch 1 controls the server and one requestor, while Dispatch 2 controls the other two requestors. Note that there is still only one Dir-Sync server, not two. M N M Server Requestor 1 Requestor 2 \ / / \ / / Dispatch 1 Dispatch 2 External 1 External 2 Modem 1 <-------------> Modem 2 \ \ N Requestor 3 The command line for running the Dispatch program and the External Mail program or MTA on both networks is the same for both computers: dispatch -dmn -i"external -dmn" D. INDIRECT ROUTING =================== In the following routing, Requestors 3 and 4 are indirect via Requestor 2. As with the modem connection, the Dispatch program cannot affect the indirect postoffices. A second copy of the Dispatch program must be running in order for Dir-Sync to work. M N M Server Requester 1 Requestor 3 \ / / \ / / Dispatch 1 Dispatch 2 External 1 External 2 \ / \ \ / \ \ / \ O N Requestor 2 Requester 4 The command line for the computer running Dispatch 1 and External 1 or the MTA 1 is: dispatch -dmo -i"external -dmo -a" The command line for the computer running Dispatch 2 and External 2 or the MTA 2 is: dispatch -dmn -i"external -dmo -a" THE DIR-SYNC PROCESS ==================== Dir-Sync can be divided into four distinct times. Time 0 (T0) uses the Administrator program to create or modify a user or group that is participating in Dir-Sync. When the user or group is created or modified, a record is added to the REQTRANS.GLB file. The REQTRANS.GLB file is the entry point for all Dir-Sync names. Time 1 (T1) is the Send Updates time, when the requestors create mail messages for the Dir-Sync server. These messages contain the status of the requestor and any updates of the requestor's address list that are sent to the server. Time 2 (T2) is the Process Updates time for the server, when the server takes all the updates from all the requestors, adds them to the master transaction list, and creates mail messages that are sent to the requestors with the updates each requestor needs to have an updated GAL. Time 3 (T3) is the Process Updates time for the requestors, when each requestor takes the updates from the server and incorporates them into its local GAL. TIME 1 (T1) =========== T1 steps are: ------------- 1. Dispatch polls the PROCESS.GLB file at each postoffice to determine when to proceed with Dir-Sync. 2. When T1 has been reached or exceeded, dispatch spawns NSDA -RT. 3. The Nsda program in turn spawns REQMAIN -T to transmit the updates. 4. The Reqmain program opens REQCONF.GLB to get the requestor's current synchronization numbers. 5. The Reqmain program creates a mail message and an attachment, which contains the current Dir-Sync status of the requestor, and places them in the P1 directory for the server. 6. The Reqmain program reads the REQTRANS.GLB file to see if any additions or changes to the requestor's address list are waiting to be sent to the server. 7. If the REQTRANS.GLB file contains any entries, the Reqmain program creates a second mail message and an attachment, which contains the entries from REQTRANS.GLB, and places them in the P1 directory. 8. The Reqmain program writes the status of the sent updates to the DIRSYNC.LOG file. Because the mail messages for the server are in the P1 directory, the External Mail program or MTA must run to first move them into the mailbag for the Dir-Sync server and then actually deliver them to the Dir-Sync server. TIME 2 (T2) =========== T2 steps are: ------------- 1. The Dispatch program polls the PROCESS.GLB file again. 2. When T2 has been reached or exceeded, the Dispatch program spawns NSDA -S on the Dir-Sync server only. 3. The Nsda program spawns SRVMAIN -R to receive the updates. 4. The Srvmain program reads the Dir-Sync mail messages (the .MBG file for the Dir-Sync mail is the SYSTEM.MBG) and incorporates them into the MSTTRANS.GLB file. MSTTRANS.GLB is the master list of all directories and all changes that are sent to the Dir-Sync server. This list is routinely purged of old directory entries. Set the time to purge in the Administrator program by selecting Config, Dir- Sync, Server, Options, Keep Updates. 5. After incorporating the updates into the MSTTRANS.GLB file, the Srvmain program updates the SRVCONF.GLB file to reflect the current status of the Dir-Sync process. 6. The Nsda program spawns SRVMAIN -T to transmit updates to the requestors. 7. The Srvmain program reads the SRVCONF.GLB file for the requestor's sync numbers. A message and attachment are created that contain the status report from the server to the Dir-Sync administrator. The Srvmain program places the message in the P1 directory for later delivery. 8. The Srvmain program reads the MSTTRANS.GLB file and creates Dir- Sync messages that contain the entries in MSTTRANS.GLB that are numbered greater than the sync number stored in SRVCONF.GLB for that requestor. If templates are exchanged, the Srvmain program reads the various .TPL and .INF files and incorporates the template information into the attachments. The Srvmain program places these messages, like the ones during T1, in the P1 directory for later delivery to the requestors. The External Mail program or MTA is required again to transfer the Dir- Sync messages from the P1 directory to the outbound queues for the requestor postoffices and then to deliver the mail to the requestors. TIME 3 (T3) =========== T3 is the most involved step in the entire Dir-Sync cycle. It is at T3 that the Dir-Sync updates are received and the GAL and the GALINDEX.GLB file are rebuilt from scratch. This step is involved because of the time required to rebuild the various files. For large GALs (for example, more than 100,000 names), the rebuild process can take several hours. Even for small GALs (about 10,000 names) the rebuild process can take almost an hour to complete, depending on the type of computer used to run the Dispatch program. T3 steps are: ------------- 1. Dispatch polls the PROCESS.GLB file again. 2. When T3 has been reached or exceeded, Dispatch spawns NSDA -RR on each postoffice. 3. The Nsda program in turn spawns REQMAIN -R to receive the updates. 4. The Reqmain program reads the SYSTEM.MBG file for Dir-Sync mail, reads the attachments and transfers the names to the SRVTRANS.GLB file, and then exits. 5. The Nsda program spawns Import to move the names in the SRVTRANS.GLB file into the various .USR files (the name list for each postoffice). Import does this by reading the SRVTRANS.GLB file and then sorting the names into three separate queues (USRTRANS.GLB, NMETRANS.GLB, and GWTRANS.GLB). USRTRANS.GLB holds names that are network/postoffice type names (SNADS, PROFS, OV). NMETRANS.GLB holds PC Mail names that are to be deleted or modified. GWTRANS.GLB holds gateway names. 6. If template exchange is enabled, each postoffice template and information file (.TPL and .INF) is read. If templates are enabled but no templates have been sent to the requestor postoffice, the warning about no templates is generated. This is a very common warning but it is not critical to Dir-Sync. 7. The Import utility combines the names from the temporary queues and the templates into the .USR or .NME files. Then it clears the queues and exits. 8. The Nsda program spawns the Rebuild utility. 9. The Rebuild utility reads the local postoffice list (ADMIN.NME), each gateway's .NME list, the network list, and each external postoffice's .USR list into a temporary GAL. This temporary file is sorted and a temporary GAL index is built. This rebuild step is by far the most time-consuming process. 10. The Rebuild utility deletes the old GAL and GAL index files, renames the temporary GAL index to GALINDEX.GLB, and renames the temporary GAL to GAL.NME. 11. The Rebuild Utility places all external postoffice and gateway names in the GALNETPO.GLB file. This file is used to speed up the time required to resolve an external name to a corresponding mailbag. 12. The Rebuild utility exits, the Nsda program exits, and the Dir- Sync cycle is complete. GATEWAY NAMES AND DIR-SYNC ========================== It is possible to use Dir-Sync to distribute gateway names to all the requestor postoffices. However, no version 2.x or 3.0 gateways directly support Dir-Sync as a registered requestor, so the process to place gateway names into the Dir-Sync cycle is one that the postoffice administrator must do manually. Each requestor postoffice must have either the gateway or the access (also known as downstream) component of the gateway installed. When this is done, use the Administrator program Config, Dir-Sync, Requestor, Types command to set the postoffice to accept the gateway names. Normally, the Types option only has Microsoft Mail enabled, but installing the gateway places the gateway type into the Types list. The administrator can select the gateway and respond Yes; then the gateway names will be placed in the GAL. Next, create a text file that contains the gateway names to be entered into Dir-Sync. Use the import format described in the "The Import Utility" section of the "Microsoft Mail Administrator's Guide." On any requestor postoffice, run the Import utility as follows: import admin -p -e -f The -E option makes the Import utility add the names in to the REQTRANS.GLB file instead of the gateway name file, as would occur without the -E option. The next Dir-Sync cycle will take these names, as with regular PC Mail names, and send them to the server for incorporation into all the requestor GALs. MANUAL DIR-SYNC =============== At times it may be necessary to check on the status of Dir-Sync and verify that all parts and executables are running correctly. A manual Dir-Sync cycle can accomplish this. Manual Dir-Sync follows the same process as the Dispatch program follows, so similar notation is used below. The indented lines are command lines that the system administrator must enter. SETUP ===== Turn off all External Mail programs or MTAs, if possible. Investigating the status of Dir-Sync will be much easier if no Externals or MTAs are running during the manual Dir-Sync cycle. Add or modify a user on each postoffice so you can follow the flow of Dir-Sync mail from the requestors to the server and back again. TIME 1 (T1) =========== 1. On every postoffice involved in Dir-Sync, including the server postoffice, type: reqmain -t -d This step generates the outgoing Dir-Sync mail message and places it in the P1 directory. 2. On the computer that all the postoffices are mapped to, type: external -0 -d NOTE: The -0 option is a zero, not the letter O. This step transfers the mail into the outbound queue for the Dir- Sync server. If an MTA such as a gateway is used to transfer mail between postoffices, use the MTA command instead of the External Mail command. Use the Administrator program on each requestor postoffice to check the queue for the Dir-Sync server postoffice. There should be at least two $SYSTEM messages in the queue. One will be a status report, the other will have a header that looks like the following $SYSTEM ReqTx=## (was ##), S = ##, I = ## ## sent where: ReqTx Is the current requestor sync number. (was) Is the old requestor sync number. S Is the requestor's sync number for the server. I Is the number of imports requested to date. ## sent Is the number of address updates contained in the attachment. NOTE: If another External Mail program or MTA is still active, the mail may have been delivered before the queue was checked. 3. On the computer that all the postoffices are mapped to, type: external -0 -d This invocation of the External Mail program actually delivers the outgoing Dir-Sync mail. 4. On the Dir-Sync server, type: srvmain -r srvmain -t The SRVMAIN -R command receives the updates and places them in the MSTTRANS.GLB file. The SRVMAIN -T command reads the SRVCONF.GLB file, determines which updates need to be sent to each requestor postoffice, and places the outgoing mail into the P1 directory. 5. On the computer that all the postoffices are mapped to, type: external -0 -d This again moves the mail to the outbound queues. The server's queues for the requestor postoffices should show a $SYSTEM message $SYSTEM SrvTx R=## S=## (was ##) I = ## ## sent where: R Is the new requestor sync number that was sent from the requestor. S Is the new server sync number for the requestor. (was) Is the old server sync number. I Is the number of imports requested to date. ## sent Is the number of updates sent back to the requestor. 6. On the computer that all the postoffices are mapped to, type: external -0 -d This delivers the updates to the requestor postoffices. 7. On every postoffice involved in Dir-Sync, type: reqmain -r -d import admin -p -q -y -d rebuild -f -d The REQMAIN -R command receives the updates and moves them into the SRVTRANS.GLB file. The IMPORT -Q command takes the SRVTRANS.GLB records and moves them into the temporary transaction files depending on the type of address they are. Then it moves the names into the USR and NME files. The -Y option tells the Import command to continue the import without asking for permission to continue. The -P parameter is the administrator's password for that postoffice. The Rebuild utility takes the network names, the .NME files, and the .USR files to build the GAL and the GAL index file. The -F option is the same as the -Y option for import. This concludes the manual Dir-Sync cycle. DIR-SYNC UTILITIES ================== There are two utilities (LISTDS.EXE and LISTQ.EXE) included with this application note that are designed to explore the various Dir-Sync files. These utilities are intended only for the system administrator and are not useful for any other user. LISTDS.EXE ========== Summary ------- The Listds utility (LISTDS.EXE) displays the server configuration file (SRVCONF.GLB) and the requestor configuration file (REQCONF.GLB). It can also reset the REQCONF.GLB file and partially reset the SRVCONF.GLB file. Syntax ------ listds -d [-r or -s] [-z []] Where: -d Is the postoffice drive. -r Shows the contents of REQCONF.GLB. -s Shows the contents of SRVCONF.GLB. -z Resets the configuration file. If this option is used, the network name, postoffice name, and Dir- Sync password, if defined, must be provided as well. Resetting the REQCONF.GLB file sets the reqsync, srvsync, and import numbers to 0. Resetting the SRVCONF.GLB file resets only the server header record; individual requestor records contained in the SRVCONF.GLB file are not reset. Example ------- The REQCONF.GLB file has the following format: Microsoft (R) Mail List DirSync Objects Utility V3.0.4000 Copyright 1991-1992 Microsoft Corporation. All rights reserved. [] not specified. Requestor Config Record Version=1, ReqSync=29, SrvSync=42, ImportSync=1, SrvTransPos=0, Limit=0 NET1/PO1, password= Requestor Directory Fields Flags=7f=GAL, PAL, TxUpdates, RxUpdates, AutoCreatePO,TxTemplate, RxTemplate, Address Types=2082 UserPos=-1, GwPos=-1, NmePos=-1 LastExport=726514913=Fri Jan 08 09:41:53 1993 LastImport=726517961=Fri Jan 08 10:32:41 1993 The fields in the REQCONF.GLB record have the following format: Field Description ------------------------------------------------------------------ Version Is the version of Dir-Sync, which is 1 in Microsoft Mail for PC Networks, version 3.0. ReqSync Is the current requestor's synchronization number. SrvSync Is the requestor's current server synchronization number. ImportSync Is the number of imports requested to date. Limit Shows how much Dir-Sync information the requestor is willing to accept at one session. 0 indicates no limit. NET1/PO1 Is the registered server postoffice for this example. The PASSWORD= parameter is the Dir-Sync password for the requestor. Flags Shows which settings the requestor has enabled. This is set with the Admin, Config, Dir-Sync, Requestor, Options command. Address Shows which address formats the requestor is Types accepting. This is set with the Admin, Config, Dir-Sync, Requestor, Types command. The other fields are used internally by the Dir-Sync process. The SRVCONF.GLB file has the following format: Microsoft (R) Mail List DirSync Objects Utility V3.0.4000 Copyright 1991-1992 Microsoft Corporation. All rights reserved. [] not specified. Server Config Record Version=1, Updates=0, UpdatesPos=-1, Sync=42, SyncCycle=3, SyncWindow=14 Admin mailbox=admin, Keep window=14 Server Requestor Record #1 ACTIVE Name=PCM:NET1/PO1, Address=(2)NET1/PO1/$SYSTEM, Password= ReqSync=29, SrvSync=42, ImportSync=1, SyncCycle=3 Last call=726703795=Sun Jan 10 14:09:55 1993 Flags=2A=EXPORTCONF,UPDATEREQ,TEMPLATE,INTERNAL Request Type=2082,Import Type=2,MstTrans=4903, Limit=0, No ImportPending Server Requestor Record #2 ACTIVE Name=PCM:NET1/PO2, Address=(2)NET1/PO2/$SYSTEM, Password= ReqSync=20, SrvSync=42, ImportSync=1, SyncCycle=3 Last call=726703795=Sun Jan 10 14:09:55 1993 Flags=2A=EXPORTCONF,UPDATEREQ,TEMPLATE,INTERNAL Request Type=2082,Import Type=2,MstTrans=4903, Limit=0, No ImportPending The first block is the header information: Field Description ----------------------------------------------------------------- Version=1 Is for Mail 3.0. Updates Is not used and should always be 0 for Mail 3.0. Sync Is the current server synchronization number. SyncCycle Is the number of Dir-Sync cycles completed since the SRVCONF.GLB record was created. Admin Is the person to whom Dir-Sync status messages mailbox are sent. Keep Window Is the number of days updates are kept before being discarded. Following the header block should be one record for each requestor registered with the server. The first record should always be the record for the server itself. The fields in the requestor record are: Field Description ------------------------------------------------------------------ Name Is the requestor's type (PCM is PC Mail), network, and postoffice name. Address Is the specific address Dir-Sync mail goes to. $System is the special address for system mail. Password Is the Dir-Sync password for the requestor. This must match the password recorded in the REQCONF.GLB file for that requestor. ReqSync Is the current requestor's synchronization number. This value should be the same as that stored in the SRVCONF.GLB file SrvSync Is the requestor's current server synchronization number. This value should be the same as that stored in the SRVCONF.GLB file. The other fields are used internally by the Dir-Sync process. <>