Changing the Solution Directory

Normally, TDI is installed to a program area (e.g. Program Files) and the TDI project files are located in the Solution Directory, which under Windows is "My Documents/TDI"; In the same way that you store your text processor binaries separately from the documents you write with them. This making it easier to include TDI solutions in your existing backup plan.

I've seen a lot of people configure TDI to use the installation folder as the solution directory. Understandable. When I first install some new piece of software, I also want to limit it's spread on my disk until I've decided whether that to let it stay or not.

So when that time comes when you want to change this choice of Solution Directory, then you only have to edit a couple of batch/script files:

o ibmdisrv - which starts the TDI Server.
o ibmditk - for the Config Editor dev environment).

Both are either batch-files or scripts and reside in the TDI install folder.

Just change the third line shown in the snippet below for both files (Windows example):

   . . .
rem Only set TDI_SOLDIR if it hasn't been set already in caller's shell
if .%TDI_SOLDIR%==. (
   set TDI_SOLDIR="C:\Documents and Settings\\My Documents\TDI"
)
call "C:\Program Files\IBM\TDI\V6.1.1_GA\ibmdicwd" %TDI_SOLDIR%
   . . .

As you can see, there is also a Windows environment variable (TDI_SOLDIR) that you can use as well to change this setting.

And finally, there is the -s commandline option for specifying the Solution Directory on startup of either the Server or the CE:

  ibmdisrv -c ITIL.xml -r Omnibus_2_TSRM, TSRM_2_Omnibus -s d:\ISM\

This is also a handy option if you need to run someone else's Config, but don't want to mix their project files in with yours. Or if you want to have multiple TDI Servers running on the same machine but with different API ports. Multiple SolDirs means multiple solution.properties files, each with its own api.remote.naming.port value.

P.S. I don't have the Unix scripts in front of me right now, but they are also easy to fix.

Newsgroup

The TDI NNTP newsgroup has been out-of-order for a while now. There is a Google Group that mirrors all its content, plus has a good deal of its own activity:

http://groups.google.com/group/ibm.software.network.directory-integrator/topics?lnk=li

It also has good spam filtering.

See you in the forums!

Sharing Secrets, aka Synchronizing Passwords

There are plenty of good reasons for synchronizing credentials, like migrating a Portal from one identity store to another, or as an alternative to single sign-on - such as making sure that the Windows password gets put into (for example) an LDAP directory. But while transferring user id's is pretty straightforward, synchronizing passwords is not. This is because passwords are encrypted information, and typically encrypted using a one-way algorithm. So even if you can read the password out of a data store, it will be a binary value that cannot be copied as-is to another authentication system. This pretty much prevents you from using standard change detection and propagation techniques.

Unless of course both the source and target systems employ the exact same encryption algorithm. In this case you should be able to read in the password attribute and then write this binary value to the target. Note that the target system must also support doing a direct write to this attributes -- i.e. without encrypting it again. Note also that in order to read in an attribute with a binary value (like userPassword ) using the TDI LDAP Connector, you must first list it in the Binary Attributes parameter.

There are also technologies available for catching passwords in clear text, when they are changed. TDI provides plugins for a number of popular identity stores, like IBM Tivoli Directory Server, Domino (HTTP InternetPassword only), Microsoft Active Directory and SunOne. These plugins securely capture passwords, making them available for processing by TDI AssemblyLines.

But that only deals with passwords as they are changed, so we're back to the original question: how to synchronize (or migrate) existing, already-encrypted passwords?

That's what I love about TDI: if one path is blocked (e.g. decrypting passwords) then pick a protocol or api and try another route. So in this case, instead of trying to pull passwords out of a system, hook TDI into the login/authentication mechanisms and let clear-text passwords be pushed to your AssemblyLines.

One example is with Tivoli Access Manager for e-Business. TAMeb can be configured to securely call out to TDI with id/password during authentication. This not only allows TDI to check these credentials against multiple identity stores thereby extending the configurability of the authentication service, but it lets you write these clear-text passwords to any number of targets, provisioning users on-the-fly with the same set of credentials. The end result is password synchronization on demand as these credentials are applied.

I've also seen similar solutions where callouts were made directly from the Portal login service, again providing the same multi-backend capability plus the opportunity for sync'ing passwords. One creative solution even re-directed the login page to a TDI AssemblyLine using the HTTP Server Connector (mini web server). TDI provided the login page, verifying (and sync'ing) credentials, directing the connection back to Portal once the user was authenticated.

So while you can't pry encrypted passwords out of a store, there are ways of working around this limitation -- given the right tools :)


Addendum: Borrowed from a TDI newgroup post by Christian Chateauvieux (of Metamerge and IBM fame) outlining another solution to this challenge: The Tivoli Directory Server pass-through authentication plugin, shipped with TDS 6.1.

This is how it works: On an authentication failure, TDS can be configured to call out to the plugin which attempts to authenticate the same user in another directory. If the password is validated, TDS stores it so that it won't have to look it up next time. Cool solution, as long as the cached passwords are invalidated regularly enough. Or you synchronize them when they are changed.


Addendum II: You could extend this pass-through feature to support multiple backends, or RDBMS's, Notes db's and even flat files. Simply use the TDI LDAP Server Connector to build a simple LDAP simulator AssemblyLine. Just catch the LDAP bind/rebind operations, then check the credentials (and/or migrate them) against whatever backends you want using the appropriate Connectors and Java calls.

If you want TDI to return mutliple values for an Attribute then you
just need to return a JavaScript array:

   var mValues = Array();
   mValues[0] = "first one";
   mValues[1] = "second one";
   mValues[2] = "you get the picture";

   ret.value = mValues;

This would of course be in a scripted Attribute Map.

If it's just literal values you want to return (like objectClass) you can easily do it like this:

   ret.value = ["top", 
                "person", 
                "organizationalPersion", 
                "inetOrgPerson", 
                "dominoPerson"];

This method does not work when you are adding Attributes to an Entry directly from script. Then you need to either add values to an Attribute and then put it in the Entry:

   myAtt = system.newAttribute("Selection");
   myAtt.addValue("eenie");
   myAtt.addValue("meenie");
   myAtt.addValue("mynie moe");
   work.setAttribute(myAtt); // works with any Entry

...or you add the Attribute and then the values to the Entry:

   ent = system.newEntry();
   ent.setAttribute("Selection","eenie");
   ent.addAttributeValue("Selection","meenie");
   ent.addAttributeValue("Selection","mynie moe");

If instead you want to take a multi-valued Attribute and return it as a comma separated string (as some APIs require), then you could do this using an Attribute Loop and a couple of AttMap components. However, the easiest way is with a snippet of script:

   valStr = "";
   for (i = 0; i < myMultiVarAttribute.size(); i++)
         valStr += ";" + myMultiVarAttribute.getValue(i);
   ret.value = valStr.substring(1); // get rid of the first ";"

If you need the values quoted then the script would be:

   valStr = "";
   for (i = 0; i < myMultiVarAttribute.size(); i++)
         valStr += ";\"" + myMultiVarAttribute.getValue(i) + "\"";
   ret.value = valStr.substring(1); // get rid of the first ";"

Of course, there could be a double quote (") in the value itself, in which case
you'd need to know how the target system wants this encoded. Perhaps simply by
wrapping the value in single quotes(?) I'd ask Google :)

What do JDBC commit/rollback and LDAP rebind have in common?

Both are features of their respective protocols and both are available for use from script. All you have to do is get hold of the Connector Interface, aka the "CI".

var ci = thisConnector.getConnector();

Note that "thisConnector" is a handy variable that always references the current component. Also, for TDI versions prior to 6.1.1 you don't have the getConnector() method and must reference the connector member field directly:

var ci = thisConnector.connector;

Once you have the CI, you have direct access to technology/vendor/platform-specific functionality, as well as the standard methods that all CI's must implement in order to support Connector Modes. However, working with the CI directly will not invoke any Hook flows.

Hook flow execution is initiated when an "AL Component" method is called, like getnext() or lookup(). These standard AL component functions can be found in the TDI JavaDocs (Help > Low Level API) . If look at the class com.ibm.di.server.AssemblyLineComponent you'll see these calls. For example, the update() method invokes the Update Mode Hook flow which performs a a Lookup and then branches to either Add or Modify. The actual read and write operations are provided by calling methods in the attached CI: findEntry(), putEntry() and modEntry(), in that order. Calling any of these CI functions directly from script means bypassing the Hook flow logic provided by the AL Component.

In addition to the methods required to support desired Modes, a CI can contain any number of supplementary functions. The JDBCConnector class offers jdbc-specific calls like commit() and rollback(); The LDAPConnector lets you rebind() and getServerInfo().

But the fun doesn't stop here. Many CI's can also return a handle to the underlying system and vendor libraries. For example, the JDBCConnector offers a getConnection() call to return the underlying driver's Connection object, implemented according to this standard interface:

http://java.sun.com/j2se/1.5/docs/api/java/sql/Connection.html

plus vendor-specific additions.

Another example is the Notes Connector that lets you get a reference to the currently opened Domino Session, Database or View object. Then you can make direct Domino API calls to do stuff like re-certify users or invoke AdminP processes.

And, of course, Functions and Parsers have Interfaces too.

Made-to-order music videos

I found this very cool online service and made TDI ready for MTV ;)

BlueGlue remix #2

Guess I'll need to add a concert t-shirt here as well then:

Prepare to be integrated...

Why throw multiple ALs at a single problem?

I get this one occasionally and it's a very valid question from a simplicity standpoint (a TDI mantra). However, there are good reasons for dividing a workload up among multiple AssemblyLines, and better performance is one of them -- as long as the task at hand allows for multi-threading; For example, data movement/migration/sync where the ordering of updates is not significant as long as all are processed. Here is a whitepaper that outlines a number of patterns that lend themselves to multi-AL solutions:

Performance Best Practices Paper for TDI 6.1 and 6.1.1

Another reason is to improve availability. Having several simultaneous worker ALs as mentioned above also means no single point of failure. Imagine a scenario where one or more AssemblyLines (on one or more TDI Servers) read source data and write to MQ. At the other end of this secure, persistent data pipe is another set of ALs picking up these messages on a first-come-first-serve basis. If an AssemblyLine fails then the integration service is degraded; not dead.

One oft-used multi-AL technique is to have a secondary "launcher" AL with a "while(true)" loop that starts the primary long-running AssemblyLine, waiting for it to complete - which it should never do (in theory). If for some reason the mission-critical AL stops then postmortem failure handling, including logs/alerts can be made by the calling AL; Finally, the AL can be re-started again in the next cycle of the loop.

I have also seen complex logic divided up into smaller, simpler ALs for reasons of simplicity and maintenance. The developer specs out each step in their integration flows - along with constraints, schema and invariants - and them implements these as individual ALs. This allows for incremental upgrades to a solution as well as unit testing.

Finally, entire AssemblyLines can be exposed as components (called "Adapters") facilitating sharing and reuse. You can find more on this topic here:

Introducing Adapters with Tivoli Directory Integrator 6.1