TDIing out loud

Ramblings on the paradigm-shift that is TDI.

Tuesday, June 8, 2010

Loop a Connector today

If you've never tried Connector Loops you should. Especially when you find yourself stuffing initialization scripts into your Prolog Hooks - because that's the only place where you can do something before the Feed Connector (e.g. Iterator) kicks in and reads the first entry; Or if you're parsing data entries out of Attribute values (like XML/SOAP, JSON, ...) being returned by your Feed Iterator.

Let's look at pre-Feed handling first. When your solution requires initial processing to happen before the Feed section engages, why not skip using the Feed section altogether? Instead, you implement your own Feed-Flow loop down in your AL Flow section. You do this with a Connector Loop and attach any Iterator or Lookup Connector you want to it. Now add components under the loop, mix well, and violá you have a mini assemblyline baked into your AL. How handy is that. And you can have as many as you need, with the results of one Loop feeding into the next one. You can also nest them:

FOR-EACH FileInDirectory
FOR-EACH EntryInFile

As an observant reader you're thinking, sure, but how does the inner Loop know which file to read for every cycle of the outer Loop? I'm glad you asked that.

One of the cool things about Connector Loops is their "Connector Parameters" tab where you leverage your Attribute mapping skills to configure the connection, for example mapping to the filePath parameter of a File Connector. By default the Connector initializes each time the Loop starts up, and this is perfect for the file example above. However, if the inner loop were talking to a secure LDAP or database server, the cost of negotiating the connection each time is wasteful. So you press the "More..." button and tell the Loop to skip (re)initialization and just do the query. Ok, that was a couple of cool things :)

Digressing slightly, there has been some confusion, particularly among new users, in differentiating between Iterator and Loop modes. We all learn after a while that they are conceptually the same: a search query is made and results returned. But while the Iterator gracefully deals with 0, 1 or many entries returned, Loop mode bails, leaving the handling of 0 and many to the user (through the ever-popular mandatory Hooks: On No Match and On Multiple Found). Despair no longer. The Connector Loop handles this for you. You can still code the On No Match or On Multiple Found Hooks, but these are not mandatory for the Loop. So while Lookup mode is great for checking assumptions about data uniqueness and presence, with a Connector Loop you get the best of both worlds.

Moving to the second part of this lecture: Connector Loops are also how you Iterate entries found in the value of an Attribute. The secret ingredient is the Form Entry Connector which uses a Parser to iterate over entries found in the value of its Raw Data Text parameter. So you add a Connector Loop, attach the Form Entry Connector, map the Attribute with the payload to the 'entryRawData' parameter, choose your Parser and away you go. What more could you ask for?

Seriously, write me if you have more to ask for.


Steve Williams said...


How applicable are Connector Loops within the context of a RMI based ITIM adapter?


eriksorensen said...

perfect timing for this post :) just used a connector loop together with an ldap.addAttributeValue yesterday - thanks also to one of your posts in

Eddie Hartman said...

@Steve: I know that some things do not work in the context of a TIM adapter AL. However, Connector Loops should not be one of them.

In general it's the stuff that creates new threads that is not allowed (like launching another AL).

But I have little experience using TDI with TIM :) I say give it a shot.

Eddie Hartman said...

Just a pleasure! And let me know if there are other topics of interest that I should post a ramble on.

Steve Williams said...


I eventually managed to answer my own comment regarding ITIM adaptors. I thought you might be interested to know that I used your excellent post to create a TDI Assembly Line that bends the ITIM AL model to achieve something quite useful. I have used this model to implement an AL that can reconcile 1000s of accounts on 1000s of Windows servers. To do this otherwise would be a real hassle.

Would really be interested in hearing your feedback comments and thoughts on this. I've written the high-level model all up as a posting on my blog -


Eddie Hartman said...

@Steve - I've commented right back atcha :)

Nice writeup!