TDIing out loud, ok SDIing as well

Ramblings on the paradigm-shift that is TDI.

Wednesday, December 8, 2010

TDI keyboard shortcuts

If you're like me then you're lazy. For example, I'd rather spend the 8 hours making an AssemblyLine than the 1 hour of tedious work it saved me from.

I'm also not fond of mousing about. To limit this while TDI'ing, I've opened Window > Preferences > Keys in TDI and bound the following shortcut keys in All Windows and Dialogs:

  • Ctrl+R - Run this AL
  • Ctrl+D - Debug this AL
  • Ctrl + Down Arrow - Step Over
  • Ctrl+Right Arrow - Step Into
  • Ctrl+Shift+Down Arrow - Continue
  • Ctrl+C - Stop this AL run and/or debug session
  • ...
And I cycle between AL components by typing the first letter of their name.
So my index fingers never stray far from F and J. Then I can squeeze in a few more dev iterations.

And today it's -10C again. Gulf Stream where are you?!

Tuesday, June 8, 2010

Loop a Connector today

If you've never tried Connector Loops you should. Especially when you find yourself stuffing initialization scripts into your Prolog Hooks - because that's the only place where you can do something before the Feed Connector (e.g. Iterator) kicks in and reads the first entry; Or if you're parsing data entries out of Attribute values (like XML/SOAP, JSON, ...) being returned by your Feed Iterator.

Let's look at pre-Feed handling first. When your solution requires initial processing to happen before the Feed section engages, why not skip using the Feed section altogether? Instead, you implement your own Feed-Flow loop down in your AL Flow section. You do this with a Connector Loop and attach any Iterator or Lookup Connector you want to it. Now add components under the loop, mix well, and violá you have a mini assemblyline baked into your AL. How handy is that. And you can have as many as you need, with the results of one Loop feeding into the next one. You can also nest them:

FOR-EACH FileInDirectory
FOR-EACH EntryInFile

As an observant reader you're thinking, sure, but how does the inner Loop know which file to read for every cycle of the outer Loop? I'm glad you asked that.

One of the cool things about Connector Loops is their "Connector Parameters" tab where you leverage your Attribute mapping skills to configure the connection, for example mapping to the filePath parameter of a File Connector. By default the Connector initializes each time the Loop starts up, and this is perfect for the file example above. However, if the inner loop were talking to a secure LDAP or database server, the cost of negotiating the connection each time is wasteful. So you press the "More..." button and tell the Loop to skip (re)initialization and just do the query. Ok, that was a couple of cool things :)

Digressing slightly, there has been some confusion, particularly among new users, in differentiating between Iterator and Loop modes. We all learn after a while that they are conceptually the same: a search query is made and results returned. But while the Iterator gracefully deals with 0, 1 or many entries returned, Loop mode bails, leaving the handling of 0 and many to the user (through the ever-popular mandatory Hooks: On No Match and On Multiple Found). Despair no longer. The Connector Loop handles this for you. You can still code the On No Match or On Multiple Found Hooks, but these are not mandatory for the Loop. So while Lookup mode is great for checking assumptions about data uniqueness and presence, with a Connector Loop you get the best of both worlds.

Moving to the second part of this lecture: Connector Loops are also how you Iterate entries found in the value of an Attribute. The secret ingredient is the Form Entry Connector which uses a Parser to iterate over entries found in the value of its Raw Data Text parameter. So you add a Connector Loop, attach the Form Entry Connector, map the Attribute with the payload to the 'entryRawData' parameter, choose your Parser and away you go. What more could you ask for?

Seriously, write me if you have more to ask for.

Saturday, February 6, 2010

What Is The Solution Directory?

The Solution Directory is the personal working folder for a TDI Server. Each TDI Server running on the same machine must have it's own Solution Directory. The reason for this is that a TDI Server gets it's configuration settings, including settings for the JVM, from a file in the Solution Directory named After reading the solution properties file then the Server reads the file in the etc sub-folder of the TDI installation directory. Settings in override those in

If you open in a text editor you can find two important properties that are personal to any running TDI Server:

  • The port that the Server listens to for RMI API calls: api.remote.naming.port
  • The port that the Server listens to web calls: web.server.port (dashboard, FDS, CURI and REST APIs)

This is also where you provide the paths to keystores and truststores. By default, those for the TDI Server itself are named 'testserver' and found in the Solution Directory. Those used for SSL connections are by default under the serverapi sub-folder and are called 'testadmin'. The password for 'testserver' is 'server, and for 'testadmin' is 'administrator'. To import your own certificates use keytool (found under the jvm/jre/bin sub-folder of your TDI installation directory) or the Key Manager button in the TDI CE (Configuration Editor).

If you remove all properties in that you are not explicitly setting, then a TDI version upgrade could affect settings for your Server.

Leveraging the Solution Properties folder to make your solutions more portable

The Solution Directory is also the "current path" whenever you run TDI, either the CE or the Server. As a result, if you use relative filepaths then your solution is easily moved between TDI Servers. For example, edit the Editor tab of your Resources > Properties > <Project PropertyStore> and set the Collection Path/URL to a filepath that is relative to your Solution Directory - for example: MyProject/ Also, always use forward-slashes for paths, that way your solution runs on Windows and *nix.

Once your properties are stored relative, right-click on the topmost folder of your Project in the CE Navigator View and select Properties. Choose Tivoli Directory Integrator properties at the bottom and set the Linked File parameter to a relative path that points to a config file under this same folder, like MyProjext/MyProject.xml.

Make sure you also put any other support files in this folder so you can easily move the solution by simply compressing this folder and then uncompressing it in the Solution Directory of the target TDI installation.

The ReadMe.txt file you make in your project directory could look something like this:

To install this solution:

1) Unzip to your Solution Directory;
2) Edit to configure connections
3) Run the solution with this commandline:

ibmdisrv -c ADSync/ADSync.xml -r ADSync

Violá, you have something that you can hand to the operations team for deployment, pass on to a colleague, and even publish it online (and get yourself a Metamerge pen by sending me the link).

Pointing to a specific Solution Directory

Stephen Swann pointed out to me that I need to mention the ibmdisrv and ibmditk commandline argument for specifying the Solution Directory to use: the -s argument. In additon, in those cases where TDI has been installed via automated scripts and the CE is not going to be started in order to prepare the Solution Directory, there is also the -g argument to instruct the TDI Server to do this job.

    ibmdisrv -s /MyOtherSolDir -g

The above command line will instruct the TDI Server to prepare the MyOtherSolDir folder as a Solution Directory.

Your Eclipse workspace

So the Solution Directory is a tool for making solutions portable, and it is independent of where you put your TDI 7 workspace. The workspace is where the Eclipse part of TDI organizes your source files, and the directory structure there mirrors what you see in the CE Navigator. It is not recommended that you work directly with source files, apart from tying them to a Source Management tool.

What if..?

But what if you chose the program directory or "current location" when you installed, or want to move the Solution Directory for some other reason? No problem. Go to the TDI installation folder and edit the batch-file (script) bin/defaultSolDir and change the path set there. This single line sets up the TDI_SOLDIR environment variable used further down in the launcher script. So if you want to move your Solution Directory, simply change this assignment. Note that if you craft your own scripts/batch-files, also set TDI_SOLDER at the top and then refer to this variable to invoke TDI launch scripts. This makes your own scripts easier to re-use.

Changing TDI_SOLDER is only half the job. You also have to instruct the CE where the new Solution Directory is when it automatically starts up your local TDI Server, called 'Default'. You do this by right-clicking on 'Default' in the Server view and selecting Properties and making the change here too.

Final note about the Servers view

The view is not called Servers for nothing. TDI lets you define new Servers, allowing you to start multiple Servers on your PC, or connect to those running on other platforms. You can point a Project at any Server by editing its settings (right-click > Properties). This lets you test and debug the ALs there. Each Server has its own Solution Directory settings, so using relative paths means you can build your ALs on your laptop and then run them elsewhere. You will also want to use forward slashes for filepaths, since backslash only works on Windows and TDI runs a lot of other places as well.

And while I'm on the soapbox here are some additional tips: Define Logging for where your task.logmsg() messages are written; Code Error Hooks and log enough info for manual intervention; Write status messages to the command line by using main.logmsg(); And set up Connector Auto-Reconnect (Connection Errors).