TDIing out loud, ok SDIing as well

Ramblings on the paradigm-shift that is TDI.

Friday, September 13, 2013

JSON and XML Tutorial - Part 1

If you're working with cloud services, or probably any kind of services, you're most likely working with JSON, XML (e.g. SOAP web services) or both. Although TDI provides specific components for handling web services, there is an easier approach to deal with both formats quickly and flexibly - and in the exact same way - if you're ready to do a little scripting. This is a quick tutorial to get you started with TDI's hierarchical Entry features - or hEntry for short.

As an experienced TDIer, you know that data in the system is carried around in Entry objects. Each Entry holds zero or more named Attributes. Each Attribute contains zero or more values.  Each value is a specific Java object used to represent the actual data content of that Attribute. We'll call this a 'flat Entry', since it represents a one dimensional list of Attributes - like a directory entry or a database table.

As of version 7, the Entry now has an hierarchical mode to represent a tree of data. hEntry mode is enabled automatically when you create an Entry based on hierarchical data, for example parsing XML or JSON. You can also explicitly call the enableDOM() method of an Entry, which then makes the DOM Interface functionality active and available. Or you can just add hierarchical attributes to it.

hentry = system.newEntry() // hentry is still flat
hentry.root = null // now it has a single attribute
hentry.root.branch = null // it's become hierarchical now
hentry.root.branch.leaf = "Green" // add a node with a text value

The above script creates a tree of data in the form of an hEntry. You can get the XML representation like this:

task.logmsg(hentry.toXML())

Here is the output:

<root>
  <branch>
  <leaf>Green</leaf>
  </branch>
</root>

And produce the JSON version like this:

task.logmsg(hentry.toJSON())

Which gives you the following:

{"root":{"branch":{"leaf":"Green"}}}

At this point it might help to know a little about how DOM (the XML Document Object Model) works. Here is where I learned about it: http://www.w3schools.com/dom/

In short, the DOM Model describes how an XML document is organized into a tree structure, and the DOM Interface provides methods for reading, searching and manipulating the leaves and branches of this tree. JSON is another way of describing hierarchical data. Once you have converted hierarchical data (either XML or JSON) into an hEntry then you use the DOM Interface methods to work with the data.

The Entry object provides a couple of static methods for turning hierarchical data into an hEntry: fromJSON() and fromXML(). Since they are static methods, this means you can call them using either an existing Entry object - e.g. work.fromJSON() - or by using the Class itself: com.ibm.di.entry.Entry.fromJSON(). Since you have methods for turning an hEntry into an XML or JSON representation, this makes JSON to XML conversion as simple as:

xmlString = work.fromJSON(jsonString).toXML()

And note that the above snippet will not change the contents of work. We're just making use of the static methods.

So much for the theory. You can easily play with this using the Javascript View at the bottom of your TDI Workbench, or by firing up the Debugger. Stay tuned for the next part of this discussion.

14 comments:

Herta said...

What would be the preferred way to convert an Entry object into JSON if you'r still working with TDI 7.0? If I'm not mistaking, the toJSON method was added in 7.2.

Eddie Hartman said...

@Herta Unfortunately there is no JSON functionality in TDI 7.0. The JSON Parser is first available in 7.1 Fixpack 8. You could always try to script your own toJson() function. For flat entries it should not be too challenging - if you're comfortable with Javascript.

You could also ask this question in the TDI forum to see if anyone out there has done this already.

https://groups.google.com/forum/?fromgroups#!forum/ibm.software.network.directory-integrator

Herta said...

Thank you Eddie! 7.1 is good news though, since we also run a 7.1.1 installation, just not for this project. Gives me more arguments to upgrade the project I work with.

Unknown said...

Is there a way to make the parser interpret select attributes as single element arrays?

The RestService I am with (or against :-P) expects arrays at certain points (a membership should have an array of members) which creates a problem since sometimes the membership only have one member, and the toJSON() method parses them as single entry attributes.

Eddie Hartman said...

@Herta - Well there are new Connectors for QRadar and SCIM, plus an improved ISAM Connector that does not require special jar files. Plus with the web moving towards *aaS via REST services, the JSON handling is invaluable. I often convert XML to JSON just to make handling easier, and then back to XML for the response :)

Eddie Hartman said...

@John - I'm not totally sure what you mean. Could you post an example or two, e.g. in the TDI Forum, and then I can whip up a little Javascript in response.

Unknown said...

Thank you Eddie!

For any wandering TDI:ers with similar questions, this is the link to my question on the google forum:
https://groups.google.com/forum/#!topic/ibm.software.network.directory-integrator/BE9SV_CWmbY

@Eddie, I did find sort of a "workaround" - but it gave me additional questions. :-)

Gaurav Sinha said...

Hi Eddie,

Could I please draw your attention to one of my problems at http://stackoverflow.com/questions/38713203/handle-multi-valued-json-response-in-tivoli-directory-integrator? This might not be the right place but I do see you have handled this kind of situation with TDI quite frequently and hence asking for your help.

Thanks and Regards,
Gaurav.

Eddie Hartman said...

First off, I would suggest dropping your question in the TDI Forum:

https://groups.google.com/group/ibm.software.network.directory-integrator/topics?gvc=2

And I've added to the answer already posted to stackoverflow, Gaurav :)

sachindave said...

Hi Eddie,

We are looking sample HttpClient connector to use ISIM Rest API.
Please show us some light in the world of dark IBM documentation :)

Best Regards,
Dave

Eddie Hartman said...

@sachindav - that's a tall order. I plan to finish off a post regarding how to play with REST APIs using TDI, because much of my exploratory work is done using the AL Debugger. All you need is an initialized http client connector:

http = system.getConnector("ibmdi.HTTPClient");
http.initialize(null);

Done! Then you can set up an entry which has any number of "http.*" named attributes, and these become either Headers, like "http.Accept" and "http.Authorization", or they are the method to use (GET, POST, ...) as "http.method", the url as "http.url", etc. Then you make the call by passing in the entry object. For example:

e = system.newEntry();
e["http.url"] = "http://someaddress.com/some/path";
e["http.method"] = "GET";
e["http.accept"] = "application/json";

returnEntry = http.queryReply(e);

After this point, returnEntry.getString("http.bodyAsString") will get you the response, and returnEntry.getString("http.responseCode") is the HTTP response code itself.

Just to get started :)

sachindave said...

Hi Eddie,

Thanks for your reply but it will take a while for me to understand your input as I am a newbie on TDI. :)

Best Regards,
Dave

sachindave said...

Hi Eddie,

Do i have to write this code in the BeforeInitialize or Prolog hook of httpclient connector ?
I have understand your input to some extent but I am not sure where should I write this code in TDI ?

Best Regards,
Dave

Eddie Hartman said...

Hi Dave,

You can do stuff like this in the Before Init Hook of a Connector, OR you can do it all from an Empty Script instead (which is what I do). These two lines of script set up an HTTP Client Connector and initialize it:

http = system.getConnector("ibmdi.HTTPClient");
http.initialize(null);

So after this you can use and reuse this component to make any number of requests to one or more web services.

I recommend that you create an AL with a single Empty Script in it. Then you add the lines I've written above and in my previous post. I typically add a second Empty Script that I name 'Breakpoint' so that there is always something to Step to after the last line of script code. Then fire up the AL Debugger and step through your own script. Once you have 'http' up and running and have defined your own Entry object ('e') then you can interactively experiment with your HTTP calls.