Take Some REST

Open Text Programming has gone modern now we have REST in abundance. developer.opentext.com is a self registering site and most of Content Server a.k.a Livelink is hidden very carefully under a heading. Unless you are an advanced programmer who knows how to create dummy responses to REST API Requests most of us will require a working livelink system without which the free site only serves as reference documentation. I wish the POST of the website actually worked on the other side with some carefully crafted mechanisms. In contrast look at the OTDS offering. It gives you access to do it live in one’s OTDS using curl.

The OT Developer Site

Shortcomings perhaps OT can easily fix!

Most of the time developers who know languages and have a general idea about modern technologies get frustrated very easily as they expect modern methods like Google ,Bing, Github etc. to get easy to understand examples and re-purpose. The problem with the OT REST API is it does not level set expectations at all. To draw an example consider when OT moved form a mature LAPI to the CWS Programming .There were Programming examples that included setup of one’s environment in Java or .NET, there were 3 or 4 most common tasks across Livelink a.k.a Content Server .Fast Forward to REST about 6 years earlier than before OT programmers showed use of POSTMAN and common tasks. Just like one day communities vanished OT retired the entire code snippets an voila a brand new website as above beautiful to look at but hardly useful. I cannot fully understand Swagger documentation maybe since I haven’t spend any time learning it. It is akin to looking at JavaDocs doesn’t/wouldn’t ring a bell to the “Accidental Programmers”.

Advise

SmartUI

Use of the SmartUI is the best debugger. The REST API is what the SmartUI is written in so if you know debugging with Fiddler or Network tools that is the simplest way one can progress in a timely manner. In many questions posed here when I get time all I do is look at in SmartUI and tell the forum how to do it.

POSTMAN

One cannot stress the usefulness of this tool for e.g. one can do what is OT Swagger saying with this and straightaway extrapolate the code .If modern programmers do not know this what a REST API does is send a payload(including headers) to a website(Livelink is a web application) so using HTTP one can even use TELNET to talk to the server 🙂

Implementation Notes of REST API as written in developer.opentext.com

Is it correct Yes absolutely would a novice programmer understand it ?questionable. Does everyone understand what a Header is as the entire REST is basically authenticated in other languages(software packages) by a API Token.

Creating a Node a Document

Some one in OT just took code in a Postman snippet and sanitized it ,does it tell you how to add a File ? This was not the case some time back .It had good instructions on how to do the Rest queries using POSTman.

Recent Postings that I contributed

These are links in my Personal Workspace in OT KB so you need a active license to see it.

If anyone likes to see them in Google Docs comment I can make that happen

“WF REST API EXPERIMENTS.docx” This code shows how to get attachments in a Initiated workflow. There are almost no official examples except a discussion thread that serves as a community collaborated example.

“JAVA REST API EXPERIMENTS .docx” This is code which I show that does not use any REST Clients but rely on POJO. Actually shows attaching a document through multipart-formdata.

Advertisement

Useful Hidden URL’s

OT hides many useful URL’s without providing much documentation an effort to codify what is available there

Command	Usage	Comments
admin.testargs	http://localhost/OTCS/llisapi.dll?func=admin.testargs&help	SA function that can tell you who the user is particularly helpful in debugging authentication.Also can be used to find memory leaks.REMOTE_USER will show up if the webserver has IWA enabled.
admin.threadstatus	http://localhost/OTCS/llisapi.dll?func=admin.threadstatus	can find the status of each running thread also a good way to know if a big web report is running or dead in its tracks 🙂
func=Admin.RebuildAncestors		builds both DTreeAncestors and DBrowseAncestors tables
func=Admin.RebuildAncestors&target=DTA		builds DTreeAncestors only
func=Admin.RebuildAncestors&target=DBA		builds DBrowseAncestors only
func=attributes.dump		shows a table wise listing of category data structures.Many people do not know that CATS are a Data Structure in a filing system.I added one or two lines in that code to actually print out more info.Many organizations will use this utility to create tables that can be queried by integrators. Suggested by John Simon
Columns Customization Command Configure Container Options-somewhere hidden in there is a Customization if the check box for Enable browse list column customization is not present you are stuck?func=admin.navigationconfig		OT is on a spree to hide anything that people knew ,they remove working links old posts and what not I keep forgetting administration URL’s
SOAPUI says <ErrorMessage>Cannot determine the workspace template for business object type ‘Item’, business object ID ’11R22.5GT-7:SET1′. (labblah.qellus.com:E:\OTCS16\logs\, thread 1)</ErrorMessage>		In the Configure Business Object if you don’t provide a Workspace Template .GUI will work but SOAP will fail
Default Document Type		func=doctemplates.configureDocTemplates

Copy This To That

Recently I saw a post in OT Forum that caught my eye https://forums.opentext.com/forums/discussion/comment/935976#Comment_935976 What prompted me to write this is because the OT experts also advised using some other mechanisms (XML WF Extensions and use of WR). Now, normally I like to code as well as any other user in Livelink a.k.a Content Server so why this post is because I remember when OT announced the Item Handler and Item Reference(yes they are different) several years ago I believe when Tammy Jakubowski(now GCI) was the WF Product Manager at OpenText.It was downright confusing but did it work in complicated situation boy it did. So I also asked the same questions why would you need WF Attributes and Form attributes in the workflow because they both do the same thing. But this user kept telling me that Business Workspaces apparently has a WF Step that creates BWS’s and it would only work with WF Attributes and they didn’t want their users to see the WF Interface

I replied with a simple OOB implementation where I used a simple throwaway hardcoded documents inside my WF Attachments map and using two Item Handlers Copied what a user entered in a Form Value to a Category to the throwaway Object and using a second step copied that Category Values onto a WF Attribute.

I later explained to the user in a series of simple screencaps how one would do this as that user needed hand-holding and I find the OT Online Documentation horrendously confusing 🙂 https://knowledge.opentext.com/knowledge/cs.dll?func=ll&objId=77363139&objAction=browse

Metadata for the brave

Metadata  in Livelink (Yawn ContentServer) is mostly by a Object Called Categories.A Category Object again is of subtype 131 and it can have versions.Its content or version file looks like XML and translates the data structure for use with Livelink.When some one adds a Category to an Object like Document,Folder etc  what you are essentially doing is requesting the latest copy of the Category Template(131) or it will be the Category applied to the parent container.Then it goes through an oscript/weblingo layer where it paints the structure like is it a Text? is it Mandatory and so on.Increasingly I see OT advising customers to use SETS and that too with Multiple Values.I tried to stay away from SETS in my design because it is next to impossible to write reports when there is more than one value involved 🙂

A Picture is worth a thousand words so I am showing a Category called SapCategory as you can see I have some that are very self descriptive however these terminology is probably worth understanding.In Livelink projects you will affix useful metadata{ a set of categories(131), some Classifications(199) or RM Classifications (551) }.OT has prided itself on capability of its search and indexing engine(after all OT was the first guys to index the OED and that greatly increases the work they put in this ) so all this metadata that you put and inherit it down to millions of items is supposed to find you patterns of knowledge in your organization.The xECM side of the house puts technical categories so does AGA so that in first shot looking at the object metadata one can even say what it purports to be.Classifications and RM Classifications look like category metadata but they allow alternate taxonomies and navigation structures.

MVA-Multi Valued Attribute indicate by the + and – signs.In any case if you only have one Value also it is internally a List of values.On top of it now all Livelink servers can have that in more than one language . OT does not the make category metadata multi language but our good friends at Cassia ECM make it and sell it through OT.

Set -The Set is a place holder where in you can define more and more primitive datatypes and it looks aesthetically pleasing(?) or even easier to understand data.Both Sets and Categories subtype will say AttrType=-18 and AttrID indicates the position.

For techies see my revelation to the world I think the link is permanently lost

There is a simple Request Handler called attributes.dump ( perhaps it is hidden in the admin.index pages and you might need a SA prv) you will see that the Set Identifier does not have a Attribute ID

Nobody from OpenText has told the programming world or developers how to look at this DataStructure well not in so many words at least.. Everyone is using their own methods do something.How many SET parsing/example  requests happen in the OT KB and OT answers cryptically? Or as a Java or a C# programmer is one supposed to be that intelligent to figure everything out? I believe it is very shortsighted of OT to push towards their professional services,or WebReports to achieve anything and everything.Since the move from LAPI some hobby programmers have shown the parsing.I have had countless requests from developers who wanted to see how to go about it so this is a way to help them. Actually since all ReturnValues are coded for OOP apparently this is much easier than in LAPI days but still a new programmer cannot be expected to become a livelink knowledgeable person for participating in one project.

Imagine a Node Object like Document or Folder has had a category attached,then its values are in its MetaData Object.The MetaData Object consists of AttributeGroups which are akin to List of Categories.So if you have applied 7 Categorise expect 7.These 7 will show you empty shells or filled shells of information.

So I have this and another category attached /applied to an object

As you can see those are the two categories or an AttributeGroups Object of two AttributeGroup.OK so if you isolate one AttributeGroup you will get the individual keys(Attributes).OK then you need to test/cast these complex structures from their Livelink given out DataValue to things people can understand Namely StrinValue,IntegerValue and DateValue.Now if a Set is encountered you will get a TableValue object.Now the TableValue will contain a excel row like representation and is a RowValue Object.Once you isolate the RowValue into its DataValue you will get its IntegerValue,StringValue and such like.Good thing they did not allow a Set inside a Set 🙂

The key of an Attribute is made up of <category dataid>_<version>_<attributeID>

So hopefully the code I post in C# helps a poor soul.Now please understand that the Category Data Structure is a pretty complicated way how the Data is shown,how the Data is held in a Table and how the reporting table has been considered. When we were in the LAPI era the entire structure was given to you as a LLValue Object out of which you had to get the Types isolated and program.This was me in 2004 battling out a worked out example in LAPI documentation   which actually did not work if you actually cut and pasted the sample.So I just took it as a challenge and trying to learn the OT debugger and its Script actually helped me with it

 

Link to Code File coming soon 🙂 One thing I noticed in 2017 against CS16 is now OT has already built a category called “External Att” with some commonly asked of functionality Good Job…

Hopefully they might be trying to get rid of the “Additional Node Attributes” piece in OT

Screen Cap of when I run the code against a Node in LL.You can optimize the loops when you understand how the code is supposed to be written.Frankly my intention is to show  you how  to iterate the DataStructure.

 

A C# file that I wrote that takes a NodeID in Livelink and spits out Category Info

Including Sets & MVA

 

 

Code Demonstration Pages

Mostly this will consist of CWS or SOAP samples.My setup is against a CS16 computer with OTDS but when I test I make sure I can login as ‘Admin’.For ease I write this using the Visual Studio C# express.Note Java WSAPI clients has differing Syntax,my blog has some hints but for the most part it will be adaptable.Note programmers who work in this are encoraged to understand flow of your code into the livelink server see my posts on a proper way to interpret errors you are getting in your program.when time permits I will post a a flow diagram.

What is CWS ?   How do you DEBUG ?  How to look at a SOAP Envelope?
You just have to get the shell from OT for starting java dev. Java data structures are less forgiving in their get/sets than C# as well. Common misconceptions like to write java client livelink has to run in tomcat all that is not true,the server part of livelink can be anything and the webservices hosted either way and you can write .net clients and java clients

A few years back the OT community had a lot of programmers and they all contributed widely to communities.opentext.com .Very thoughtless of OT one fine morning they shutdown the site.Most of the code I wrote for public consumption I had in my personal workspace or google drive.This time I am just going to put them in my google drive.You just have to requests me to see it.Note I will not entertain any requests for obtaining software or consultation.I do this pro bono so I don’t lose touch.

How to Rename Nodes in C#	Link	Request by
Workflow Service Example	Link	Request by
Categories/Attributes manipulation in C#	Link	Request by
Beginners parsing example to Categories/Attributes C#	Link	Request by
How do you manipulate Multi Valued Attributes( MVA) Java Snippet	Link	Answering for a KB thread
Beginners Adding a Version to a Existing File JAVA	Link	Answering for a KB thread
How to set Mandatory Attributes in a move (No Not as simple as it sounds:)	Link	Answering for a KB thread
A C# example that shows SET parsing and MVA’s	Link	Answering for a KB thread

Quickly make a LL System without all the document data

Today I wanted to learn the new Distributed Agent framework in Oscript. To do that I quickly rigged up a VM and had a  oracle export done and imported so it looked like something  with some structure.I looked at the EFS attached it was our DEV machines and it had gigs and gigs worth of data.How do I get categories form this system without copying all that trash so I wrote a query to find all files in the category

.

Notice if I could get just those files needed into my VM it would work so after I put them in Excel and derived a formula I put them in Notepad++.The xCopy with the * creates the hierarchy so that I could copy the small category version files all in all about 1.76 MB very easily.

so it created the necessary structure (So I don’t have to mess with the table entries)

after I put them under my new VM’s  file path I can see my categories fine

I am pretty sure I can do this for WF maps,form templates and such like but have to learn DA

Why Oscript solutions are not passé

Over the years in my work with OpenText Livelink I am fortunate to associate myself with like minds and who like me share their code,goodwill and advice selflessly .I cherish and hold their friendship above all professional accolades I have ever got.Recently I had lunch with Ossie Moore  Ossie’s LinkedIn it was very short and I would have loved it to last longer as we could talk shop endlessly.Most of what I write in this has bearing to our lunch and he did steer this post. Ossie  for people who do not know him is quite a character.Let’s say you had a requirement,most of us would probably settle for a pretty decent code either in oscript,lapi as that was the predominant development methods one had at the time.Not Ossie he would probably write our kind of code and before releasing it would have at least two more better implementations,he would have thought about the user coming through the web,the other interfaces LL explorer and so on.That is the difference.

Let’s look at ECM deployments in major organizations.

X Organization has identified a massive unstructured data problem.X has a new manager on the block who has had experience with a technology vendor.Insert <vendorname> here. This is how most ECM deployments happen a lot of people passionate in some technology who would solutionize the deployment to suit the vendor.Most of them are assisted by Gartner, Forrester those kinds .I started working in OT technology before these rating companies came up with a barometer of wants so I am going to continue with my cynicism.

So after X Organization will roll out a OOB approach ,people are brought in.Naturally if the product is user friendly and easy its use will grow.Simple things like user activity,data growth etc are good ways to identify its usefulness.Now most people would want to send the link of the objects hoping that it won’t break on re-organizing the structure or re parenting as most of document work revolves around.So you may want to invest on something that people have put thought on like a GUID based system(Livelink, DCTM).Frankly I know only Livelink and DCTM very little. I hate Sharepoint (we call it $carepoint) with a passion because the smallest of things are so difficult to get  done.Good UI all the world has gone for it anything complex you have to buy or build with an array of .net developers different frameworks the works.It also could be that I have only been programming in that and spending time in LL so over the years the idiosyncracies of builder,CSIDE none of these matter to me now because Oscript really has not changed every much.Oscript is just a huge possibilities language. One of my mentors very famous in OT circles John Simon would joke like this.You have a problem you try solving in Oscript.You will spend about 3 to 4 days on it later to find out that what you need is totally available,no need to kludge just poor documentation that is all.

So if you were eager to get into using livelink to its full extent do these things

1. Challenge a OT person or a OT solution marketer to what they are offering you. Challenge them extremely hard if it is offered at $$$ a user or so.That is how they make their money on licenses.
2. Challenge if the pitch is towards Oscript is bad that spoils your chances of upgrade.There’s an element of truth in that.I have seen people hacking their way into core code,weblingo such no nos that shows inexperience and desperation. Oscript is not understanding the creation of a request handler,webnodeaction or eventscripting it is an all round knowledge of the product the passion about the product.You can immediately tell if the programmer knows anything because if they start saying about new subtypes you may want to ask them what prompted them to create a subtype, because many times those people have access to the oscript tutorials and the “Hello World” of oscript introduces them to that.You really want a Oscripter who will follow what OT advises them of doing in that case it is very rarely  a bad fit.
3. Challenge if the pitched solution ends up working only in the webgui what about EC,what about SOAP,what about REST,what about Search

Oscript has years to go if the LL solution is still being used by customers.It can be learned to work wonders for you.The other aspect and full credit to Ossie here is LL software is the most open product much more open than opensource a oscripter can see the entire internals of it so people like me,John,Ossie all have found bugs and we all engage the good OT developers on our finds.Most senior and passionate oscripters continue to do so.There ‘s so much openness and welcoming in the OT programming community,people like David Templeton ,Kyle  Swidrowich and a countless amount of good people who have been striving to keep this open and getting communities to try it out for a better product.They have helped me and a lot of others and you can also get in into the action.

You just need the drive and passion to master LL that is all it takes.Well that maybe true for any technology 🙂

AddressBook

Today I just chanced to see a post where somebody could not get the famous AddressBook module working.This module was part of the Oscript documentation and is usually introducing the user to the various concepts livelink programming entails.Like a “Hello World” for Oscript :).I thought of helping the person and the person had deleted the post.which was originally here  He later removed it because many people do not want to appear wrong.However there is no harm in asking or being wrong.

 

Anyways I had done this for another friend so releasing into public domain here….

I did not fix many problems in the module so it is a challenge to the person doing it that is how I learned:)

To Undelete or Not

Many times Administrators and users do things stupidly and want to cover their tracks quickly and without creating unwanted attention.Most people working with livelink know that the application is responsible for the data integrity and referential integrity.So to do any proper reversal you would need to script something back.But again here you are at a disadvantage.If you resort to standard OT support the maximum you are going to get is “you have an error that is not easily fixable and we would recommend you to fix it with our professional services team”.To a org or a small company they do not have that many resources or money to do that,hence the administrator or  a power user turned developer would try to see if something can be done.Lo and behold the KB is consulted and most members who would like to show off will post in their comments.Now that goes for me also because there isn’t a day that goes by when I cannot respond to something.However I am very judicious when I tamper with the livelink database.That is really because I understand oscript and I see the many myriad transactions that happen when it is working,doing upgrades and such.But in many organizations people are well versed in Live Reports and many now have the Web Reports product which is an excellent alternative to certain kinds of programming or utility programs .This is where the start of the problems happen.Say you accepted a bad advice like fixing a column or data point in a table that the “learned KB people” advised,you would not probably notice this way into months or years at part of a upgrade so what you did in earnest will land you in irrecoverable or costly difficulties.So RESIST the urge to change values in OT given tables.If it is your table and you are responsible for it do so by all means.There is no problem with you playing in a VM and understanding it but just don’t think that touching llattrblobdata or llattrdata is all it requires to do category updates 🙂

That doesn’t absolve OT as a company,it is expected to do its fair share.Somethings might be

1. Perhaps with support paid the customer gets 10 free incident fixes. OT absorbs the development  cost and passes the solution to a would be  admin/dev in the organization.
2. A list of compiled easy to use reports and utilities,perhaps web reports, perhaps a compiled java app that uses REST or WSAPI or even a utility oscript module.
3. Some re-assurance from support to quell the panic and perhaps advise the customer to do another thing like tell them a deleted user/users necessarily need not be considered a panic situation.

 

I had a short stint working with Documentum and I did not know if the utilities it delivered were high priced like what OT does but it had a language called DQL (Documentum Query Language).So any guys who is knowledgeable on the schema could run reports and turn them into DQL commands which would basically honor data integrity and referential integrity for them.This is in a way WR does but it takes enormous patience and complexity to work with it which again would mean should I have developers supporting my application.I would advise WR people in OT to provide clear crisp examples 1 to 5 liners that should work with any livelink any schema,kind of a pre built library more like the canned Live Reports

 

I tested a user deletion today and tried to see what all tables would be affected so for KB users if you want to see it is here.hopefully I will find a cheap hosting provider to put my content and not in KB as I have been burned in my earlier attempts.

“Why it may not be a good idea to update livelink tables.docx” can be accessed via the following link: https://knowledge.opentext.com/knowledge/cs.dll/Properties/61862916

 

 

 

Demise of communities.opentext.com

RIP communities.opentext.com

Years of developer activity retired…. Just like that

How would a developer who does not have access to KB find anything informative?

Is that intentional?

Why wouldn’t  OT give out a fair warning to migrate content for content owners?

Much of code I kept for public consumption was here

So if anybody is looking for that write me I will try to find from local copies