TDIing out loud

Ramblings on the paradigm-shift that is TDI.

Wednesday, April 1, 2009

Scheduling when AssemblyLines run

When possible, use the scheduling mechanisms already in place: cronjobs, Windows scheduler, etc. Your TDI ALs can be started from the command line:

ibmdisrv -c MyConfig.xml -r MyAssemblyLine

That way if something goes wrong, your flow is restarted.

You can also do this in TDI using the Timer Connector (Iterator only). Here you can set up a scheduling mask. In the Flow section of your scheduling AL, you drop in AssemblyLine Function components (AL FCs) to launch other ALs.

One simple 'availability trick' for keeping an AL running is to start it (AL FC) inside a Loop that never ends. The AL FC is set to "Wait for completion" and will suspend until the called AssemblyLine stops. If it stops due to an error, you can catch this in the Error Hooks of the FC (& log it, send alerts, order pizza...). Regardless, the Loop -- which is Conditional and based on a script like this: ret.value = true; -- just executes the AL FC again, re-starting the stopped AssemblyLine.

So even in what is typically a long-running AL (like one listening to a changelog, or changes in Notes) you could set the Change Detection Iterator to timeout once in a while. Then you could do how keeping in the Scheduler AL, and restart the AssemblyLine again.

2 comments:

charles said...

Are there any best practices, war stories about how to put this in place in a medium sized Domino server environment, on Windows? We have a handful of different Notes app servers to get data from. We currently use ZMERGE (Notes API data tool) running via Lotus "Program Document" schedules as well as windows scheduler on 3 servers. On TDI 611 (thanks to the great videos, blogforums and pdfs) I have worked out a CSV export using the DIIOP connector from my workstation to get data from a NSF that's on a server, but I am not sure how to run this automatically in a solid production mode - do we set up a dedicated machine to run TDI or is it something that is safe to co-reside on each domino server?

Eddie Hartman said...

These days we are leaning towards using "outside" scheduling: e.g. Windows Scheduler, cronjob, etc. That way even if the TDI Server stops for some reason, you start up a new one with the next invocation. You could even schedule it to run using the tools you know and love in Domino, as long as you can execute a commandline.

Or you could set up an AL with the HTTP Server Connector and send it signals (POST or GET) with instructions on which AL to run. Or use the Timer Connector. But as I said above, it might be better to avoid letting your TDI Server process be a "single point of failure".

TDI is "light-footed". However, Domino is a workhorse and TDI will be stealing some of its resources. Usually in "larger" production environments then TDI is relegated to its own machine. A PC will do just fine - no muscular server required, unless your ALs are many and high-performance. That said, TDI is still sometimes installed piggy-back on an existing server.

Sorry for the wishy-washy answer, Charles, but your setup will be very dependent on the existing infrastructure, as well as your TDI solution itself.