Concensus Delimited Text Driver Documentation

Legal Notices

Concensus Consulting, LLC. makes no representations or warranties with respect to the contents or use of this documentation, and specifically disclaims any express or implied warranties of merchantability or fitness for any particular purpose. Further, Concensus Consulting, LLC. reserves the right to revise this publication and to make changes to its content, at any time, without obligation to notify any person or entity of such revisions or changes.

Further, Concensus Consulting, LLC. makes no representations or warranties with respect to any software, and specifically disclaims any express or implied warranties of merchantability or fitness for any particular purpose. Further, Concensus Consulting, LLC. reserves the right to make changes to any and all parts of Concensus Consulting software, at any time, without any obligation to notify any person or entity of such changes.

Any products or technical information provided under this Agreement may be subject to U.S. export controls and the trade laws of other countries. You agree to comply with all export control regulations and to obtain any required licenses or classification to export, re-export or import deliverables. You agree not to export or re-export to entities on the current U.S. export exclusion lists or to any embargoed or terrorist countries as specified in the U.S. export laws. You agree to not use deliverables for prohibited nuclear, missile, or chemical biological weaponry end uses.

Copyright © 2014 Concensus Consulting, LLC All rights reserved. No part of this publication may be reproduced, photocopied, stored on a retrieval system, or transmitted without the express written consent of the publisher.

Concensus Consulting, LLC

103 Foxtrot Drive
Mars, PA 16046

Overview

 

The Delimited Text Driver from Concensus Technologies can be configured to use delimited text files on the filesystem to synchronize data between the Identity Vault (IDV) and applications.  Using this driver you can synchronize data from the Identity Vault to any system that can consume delimited text files.  You can also synchronize data from systems that can generate delimited text files into Identity Manager.  

Driver Features

The Delimited Text Driver can use the local installation of Identity Manager or the Remote Loader Service.  The driver can be installed on either Linux or Windows where Identity Manager Engine or Remote Loader Service resides. 

Data Flow


Publisher Channel

  • The driver supports text files where each row or line in the file represents a single record.  
  • Object Class is not expected or accepted as one of the fields.  The driver is configured to support a single object class.  That class is specified in the configuration.
  • Event type is not expected or accepted as one of the fields.  The event type is specified in the driver configuration and each row is published to the Identity Manager using that event type.
  • The fields are separated by a delimiter.  The delimiter is specified in the driver configuration.
  • The driver binary does not calculate CN, Association or placement.  These data items can be calculated in policy on the Input Transformation.  It is recommended that the driver not be configured to set associations on objects in the vault.  Doing so creates a situations where the subscriber channel can receive a modify event which only carries the changes values rather than all values in the filter which may be required for the target application.

Subscriber Channel

  • Delimited text formatted files do not have a consistent way to represent Object Class or Event Type, these items are not written to a delimited text file.
  • Delimited text formatted files do not have a way to represent remove-value operations.  Only values to be added or modified will be written to a delimited text file.

Driver Limitations

As there is no live data system the driver connects to, the driver is limited in some of the events or commands that it handles.  The

Subscriber Channel Events

  • <add> events
  • <modify> events.  If the driver is configured with associations then the engine may submit modify events.  This event type is problematic for the driver, as only the attributes which have been modified will be present in the XML from the engine.  This will likely not be sufficient to generate a row which can be consumed by the target application.  If you choose to association and handle modify events you will need to write policy to query back to the Identity Vault to retrieve the rest of the attributes for inclusion in the XML.  Since there is no real application to associate with, it is a best practice to not write associations to objects created with this driver.
  • Driver identity query.

Publisher Channel Events

  • <add> events are supported.  However, the <add> event type on the publisher can be transformed into any other desired event in the Event Transformation policy.  The developer configuring the driver is responsible to handle any event traffic generated from the new event type.

No other events are supported on either channel.  Specifically, since the driver has no access to the complete dataset owned by the target application, the driver does not support queries on either channel.  

 

 

Driver Installation 

The driver is installed from an iso image that can be obtained from the Concensus Technologies support website:  http://support.concensus.com.  It is also required that a license be obtained from Concensus Technologies.  The driver will not start without a valid license from Concensus.  

The driver requires the following files:

FileDescriptionTarget Location
delimitedtextdrivershim.jarDriver shim binary file.

On Linux, place this file in /opt/novell/eDirectory/lib/dirxml/classes

On Windows, place this file in \Novell\IdentityManager/NDS/lib

delimitedtextdriverlicense.jarConcensus License file

On Linux, place this file in /opt/novell/eDirectory/lib/dirxml/classes

On Windows, place this file in \Novell\IdentityManager/NDS/lib

CT DText Driver.xmliManager/Designer driver configuration file.Needs to be available for import in iManager or Novell IDM Designer

 

Driver Requirements

The driver requires a supported version of Novell Identity Manager.  Currently Identity Manager 3.6.1 and 4.x are supported.  The driver is supported on Windows and Linux where Identity Manager is supported.

A base configuration requires:

  • Driver license obtained from Concensus
  • Identity Manager Engine or Remote Loader system with access to the internet*
  • iManager with the Identity Manager plugins installed

Replacing an Existing License File

Driver license files expire at the end of the contract term.  They must be replaced each year as the driver will discontinue working (No events will be lost if the driver stops) at the end of the license term.  The new license will be sent to the Technical Contact on the contract.  Once that license has been obtained the following steps should be performed:

  • Windows: 
    • Unzip the file to a temporary location i.e. c:\temp
    • Copy the gmaillicense.jar file to the DirXML lib directory and Overwrite the existing file
      • IDM 3.6.1 – C:\novell\nds\lib
      • IDM 4.0 – C:\novell\identitymanager\nds\lib
    • Restart eDirectory on the server from the Services or Control Panel
  • Linux
    • Unzip the file to a temporary location i.e. /root
    • Copy the gmaillicense.jar file to /opt/novell/eDirectory/lib/dirxml/classes/ folder and overwrite the existing jar file. 
    • Restart eDirectory – rcndsd restart

The expiration date can be seen in the driver trace at level 3 or higher:

Driver Installation

Insert the disc of the Delimited Text Driver installation media that you created into the CD-ROM or DVD drive of the computer that is running a compatible version of Identity Manager or Remote Loader.

1. From the CD root folder start the installation by executing the correct program for your workstation’s platform.

  • Windows:  windows\Concensus Identity Manager Delimited Text Driver-4.0.2-Setup.exe
  • Linux: linux\rpm –ivh concensus-DXMLDTextDriver.rpm

Driver License

The driver license file is sent to the technical contact listed on the software subscription license agreement.  The technical contact will receive a zip file for each tree the connector is licensed for (Typically a production and a test tree).  The license will expire at the end of your contract term.  To install the license use the following steps:

  • Windows: 
    • Unzip the file to a temporary location i.e. c:\temp
    • Copy the gmaillicense.jar file to the DirXML lib directory
      • IDM 3.6.1 – C:\novell\nds\lib
      • IDM 4.0 – C:\novell\identitymanager\nds\lib
  • Linux
    • Unzip the file to a temporary location i.e. /root
    • Copy the gmaillicense.jar file to /var/opt/novell/eDirectory/lib/dirxml/classes/ folder

Note:  When updating your license you must remove the old jar file from the folder prior to installing the new one.  Do not rename the old jar file.  It must be removed.

Driver Import

The driver can be imported through Designer or iManager.  Concensus prefers Designer (to download visit: https://www.novell.com/coolsolutions/dirxml/designer or off of the IDM product DVD) and will document the steps here.

  1. Launch Designer and open your project or create a new project.
  2. Create a new driver by right clicking on the      icon and choosing new/driver from the menu.
  3. From the Driver Configuration Wizard/Select Base Configuration select the  button.
  4. Then click browse and select the CT DText Driver.xml file from the DTextISO\iManager folder on your CD-ROM drive and click run to begin the import.
  5. From the Import Information Requested screen fill out the following information:

Driver Name – This is the driver name.  It defaults to CT DText Driver

    1. Authentication Password – The DText Driver does not use the Authentication Password field.
    2. Driver is local/remote – Choose remote if you are using the IDM remote loader service
    3. Driver Password – Required if you are using the remote loader.  It is good security practice to fill in this field for all of your drivers..

 

  1. Select Finish on the Import Configuration screen. 
  2. Save your project in designer
  3. The driver import is now complete.  You should continue to the driver customization section to continue your setup.
  4. Once the driver is configured you need to deploy it using designer and give the driver the correct rights in the tree.  Please refer to the Novell IDM documentation on how to use designer to deploy a driver.

Driver Customization

The IDM driver for DText can be customized using Novell iManager or Designer.  The pre-configuration file used for import is only a template.  With an understanding of Identity Manager policy and xslt you can configure the driver to do what you need to with the inbound data.  For examples please review the other Identity Manager driver configuration files and Novell Cool Solutions. 

This section will document the items in the pre-configuration file. 

Driver Properties

The Driver Properties page (Right click on the driver in designer and choose properties from the menu) contains all of the items that the driver needs to startup and connect to Google. 

Driver Configuration Tab

  • Driver Options – None
    • Field Delimiter - This parameter specifies which character the driver will use as the delimiter between fields in each record in the delimited text file.  This must be a single character.  The default value is a comma.  The Field Delimiter is a required parameter.
    • Text Qualifier -  This parameter specifies which character to use as a text qualifier.  The text qualifier is used to enclose a field value that contains delimiters as data.  
    • Field Names - This parameter contains an ordered, comma-delimited list of field names.  The driver will validate each incoming record to validate that it has the correct number of fields.  The Field Names list is required.
    • Object Class Name - This parameter specifies the name of the object class the driver subscribes and publishes.  The driver can support only one object class.  The Object Class Name parameter is required.

  • Subscriber Options
    • Output File Path – This parameter specifies the directory where the driver will write output files.
    • Output File Extension - This parameter specifies the extension the driver will use when naming output files.   
    • Maximum Number of Transactions per File - This parameter specifies the maximum number of transactions that can be written to a file.  When the driver opens an output file for writing transactions it will write transactions as they are received until this limit is reached, or until one of the two Time-based limits are reached.  Set this parameter to 0 to allow unlimited transactions to the output file.
    • Maximum Time in Seconds Before Flushing Transactions - This parameter specifies the maximum number of seconds before flushing transctions.  If the driver has an output file open and has no additional transactions to write to it the driver will close the output file when this number of seconds have passed.  Setting this parameter to 0 removes this time-based flush trigger.

      Note

      This time interval begins once the last file update has been written. It does not apply to the lifetime of the file. The timer is reset each time a new record is written. If the number of seconds specified in this parameter passes after the last write to the file, then the file will be closed.

    • Time of Day (local time) to flush transactions - Setting this parameter causes the driver to flush and close the current output file at the specified time of day.  Time can be specified at HH:MM in either 24 hour or 12 hour format.  Setting this parameter does not preclude the use of the other two file size limiting parameters.

      As long as a file is open by the subscriber channel it is not safe for opening with any other program. In order to ensure that a file is not open indefinitely one of the 3 file transaction thresholds must be set.

  • Publisher Options
    • Input File Path - This parameter specifies the directory where the driver will look for files to process.
    • Input File Extension - This parameter specifies the file extension the driver will look for in the specified directory.  The default value is .csv.
    • Rename File Extension - This parameter specifies the file extension the driver will use when renaming files once they have been processed.  The default value is .bak. 
    • Source File Character Encoding - This parameter specifies a character encoding other than the local default to use when reading files.  Leave blank for a default of the local encoding.
    • Polling Rate - This parameter specifies the number of seconds between poll cycles.
    • Publisher Heartbeat Interval – If you are using heartbeat you can set the interval in minutes.

Trace Tab 

  • Trace Level – For normal production use this value should be set to 0.  For driver testing and debug information set this to trace level 3.  Trace level 5 is used to dump more information about the driver operations. 
  • Trace file – If you are tracing you should set the path and name of the file you want to trace to. For example /var/log/dtextdriver.log.  If you set this option please be sure to set the Trace file size limit as it defaults to Unlimited. 
  • Trace file encoding – Recommended not to change from default settings
  • Trace file size limit – Typically set to no more than 1024 MB.
  • Trace name – Typically set to DText.  This is not a required entry.

Driver Filter and Schema Mapping Rules

The DText Driver can be mapped to any object class supported by eDirectory.  By default the driver is configured to synchronize User objects.  The default field list is LastName, FirstName, Title, Email, and Description.  However, this can be changed as needed.  

In order to aid in working with the Driver Filter and Schema Mapping rules the driver will respond to a request to refresh the application schema with the Object Class and Attributes the driver is configured to use.  

Generating src-dn and assocation values

By default, the driver does not know anything about the source application schema or possible values.  Therefore, there is no simple out of the box mapping for src-dn and association.  The default driver configuration has sample policies to set these attributes as part of the Input Transformation Policy.

SourceName

This policy sets the src-dn of an object described in an inbound xds document.  The sample policy contains the rule Add SourceName.  This policy verifies that it is operating on a User object.  If so, the src-dn is set to the value of FirstName and LastName concatenated together.

Generate Association

This policy sets the association value of an object described in an inbound xds document.  The sample policy contains the rule Generate Association Value.  This policy verifies that it is operating on a User object.  If so, the policy sets the association value to the value of LastName and FirstName concatenated together.

 


Appendix A - Common Driver Issues 

Issue

Example and Notes

User Placement. Do not use a leading   "\" to place users or Organization Units.

To   place a user in the root container, the dest-dn should only contain the   Username. If you are placing a user in the google Sales\Marketing container   your dest-dn should look like:

<add   class-name="User" dest-dn="Sales\Marketing\ ddare"/>

 

Organization   Units use the same format for dest-dn.

Group Placement: Do not use a   placement rule on groups as Google does not support placing groups in   organizations.

 

Unique naming: It is important that   Nicknames, Group names and usernames be unique in the Google apps domain.

When   developing a matching rule be sure to check for nicknames and usernames to   ensure proper matching. Further, naming must be unique across all Google   Organization units. It is not legal to have Sales\Marketing\ddare and   Engineering\ddare since ddare needs to be unique across the domain.

Driver Unable To Start

  1. Are the   driver jar files installed and eDirectory restarted?
  2. Have   you created the admin account in Google and logged into the web interface at   least once?