Demise of communities.opentext.com

RIP communities.opentext.com

Years of developer activity retired…. Just like that

How would a developer who does not have access to KB find anything informative?

Is that intentional?

Why wouldn’t  OT give out a fair warning to migrate content for content owners?

Much of code I kept for public consumption was here

So if anybody is looking for that write me I will try to find from local copies

 

 

General Help Series 7 -Upgrading a Livelink System The 10.5 Part

Start with my first article in this series

BTW this is kind of a cookery show kind where they put the main course in the oven and take another finished one .I did not accomplish any of this in one day it took a lot of homework and trial runs.I read the case studies by the great Rob Coutts many times over.

For sanity we decided to use a new box for CS10.5

1. Created a CS10.5 system added all the modules that we needed and connected to a dummy database and dummy EFS. Just for kicks we added some that did not exist in 971 like classifications and recman(forward thinking).Rudimentary checks to ascertain base functionality.Note all hot fixes ,patches relevant were also put.
2. Cloned this server to 3 others for Front Ends and Agents.Just architectural things as after the LL installer creates the services just copy the OTHOME over ,give it distinct names etc.Sometimes if your copies are done with IIS and or any of livelink services running sometime corrupt dll copies end up so make sure you follow proper protocol.In most cases you do not need to run the optional modules installer on your 2nd to n CS servers.Most livelink needed dll’s oscript code will try to push it to windows systems folders on startup every time.
3. Saved this and copied it as backups.
4. It is worth verifying the DB type requirements for 10.5 for e,g your 971 DB might have been 11.2.0,2 but Cs10.5 need 11.2.0.4.these are database chores that your DBA should know. Always give lots of oracle memory like SGA etc .OT has a tech article on how it is to be done.
5. Connected the binary to the prepped up 971 database.one of the first screens was the 971 box for Admin server,changed it to the new box .
6. Wanted re-starts it is a good sign.One thing I do is since I know the DB Upgrade is done by a single thread I made the threads 8 on it.If you have lots of threads they indirectly add to the Oracle load.I also do my upgrades with debug=2 and wantlogs=true.It is almost impossible to live without BareTail or a good ASCII Editor.
7. The other threads on it will just produce a warning message that the “upgrade is in progress”.
8. Once the upgrade starts you should see the thread<nn>.out file issue oscript commands and its corresponding connect<nn>.out doing DB work.Your heart will rejoice if you have them on baretail 🙂 In my case a core upgrade will move my 971 schema(6.0.8) to CS10.5 (6.2.58).If it does that it is a successful upgrade. Times will vary depending on data and content and horsepower of your DB.Do not try to run arithmetic on the numbers does not mean continuous although looking at the db upgrade log one can see the steps.The places it breaks clues OT and even if you know Oscript you can chase a lot f them.In my case it was let’s say a very smooth upgrade.
9. Since we had optional modules after the core upgrade all optional module schemas were introduced or upgraded.You should see it in bold letters in the pre upgrade page. “Your Content Server schema will Upgrade from <nn> to <nn>”.Classifications module will be introduced,Recman would be introduced(New things), ADN will upgrade from <nn> to <nn> .BTW I had to downgrade A D N to a lower one because the latest and greatest won’t cleanly co operate with the upgrade.You will not notice the difficulty if your database is new.
10. The reason why OT says to upgrade OPT modules after core upgrade is it is easier for them to pinpoint the failure.
11. The way livelink code works every time you re-start is it will read Opentext.ini and one of the first modules that loads is the DBWIZAPI .It will first try to ascertain whether core schema is what the binary says its is otherwise it will force a DB Upgrade.Before releasing the software to listen to request on 2099(suggested default),it will enumerate the module section ini to the schema section INI. So if one box had a schema aware module such as form and its INI said 2.0.4 and this box’s form module said 2.0.3 it will trigger the familiar error “You have blah blah in database but blah blah module is lower/higher”.That is the whole reason why experienced people and nowadays OT says get one server(Anchor) done correctly and clone that on other boxes.So whether you install opt modules after or before livelink code always checks this every time you re-start.
12. The Admin Service left running on any box is no problem.It contains java code to talk to the search server.Also memcached is dependent on Admin Service.I don’t really know what cluster agent does  but it is OT’s answer to smart patching.Once you resister a Admin server to the database that is when the box can be used for certain things like augmenting search.
13. We had to create a search as advised by OT so that items would come out indexed faster.I am always amazed at the search code but unfortunately one cannot see it completely as Oscript just talks to the java code.Perhaps if I had time and code was de compilable( I seriously doubt it,I have a feeling java is talking to C++ compiled code internally,how else would it scale so well).To have people always complain about search ….
14. I installed all our custom modules created clones of this and called it many different roles and we were pleasantly done
15. The rest was mainly releasing it to customers to test have their suggestions and voila…

General Help Series 6 -Upgrading a Livelink System the 971 Part

Recently I was tasked with  upgrading our systems from its initial version to two times jump so 971 to 10.5.I will chronicle my experiences here. Perhaps provide a structure.I also want to quell some myths about the whole process just to make sure you can do this logically and correctly.

First create a draft plan,in my draft plan I included this.I would need at least one VM that can house or Production 971 Binaries,a clone of the database and a clone of the EFS.If one had been storing items in perhaps Archive Server one should have a playing copy of that also. An OTAS based livelink upgrade can pose its challenges if not done correctly but it is no different than a EFS if you do it wrong you can write test data into your productive store 🙂  This involves talking to other people like DBA’s and Storage folks as a heavily used LL system can have tons and tons of data.You will most likely encounter the after effects of long time use,different administrators and their styles etc.So after a few failed  attempts I understood why OT says a DB5 verification is very important.The Verification basically runs single threaded and will try to pinpoint anomalies in the database.Most likely it will pinpoint content loss such as bulk imports etc without actual version files et al .In my case the database had wrong pointers,duplicate dataid’s (wow),removed KUAf Id’s(somebody had fun with a oracle tool).So after going back and forth between OT and us I just thought like this and created a program(oscript) to check important structures.I list 3,your mileage will vary but essentially I grabbed all categories ,form templates and wf map structures.None of these checks happen in DB5 it just looks for existence.In hilarious cases I uncovered a category data structure mysteriously would turn up as a drawing PDF.One could argue that the users would have noticed, so this was an old category that nobody noticed.You don’t have to do this but each of those anomalies would prevent a upgrade and it is back and forth with OT support so I did it just to get some time advantage.

The second thing was I had to get all the optional modules  and our system is no different from any other system a lot of optional modules some inserting schema some not.

The third thing was our systems were customized with Oscript to provide enhanced user experience  as well as awesome home written modules that takes the tedium of long running things like category upgrades,permission pushes et al a very intelligent add on to the OI module that people can use generically to upload their data like that.So our team sat down and thought of what we could retire and what we should re-code,refactor so many of them were thrown out .Many useful ones we kept.We particularly liked the Distributed Agent framework,ours is very similar to that but if time comes and there is an appetite we would code it for that.

About the re-coding effort here’s the clunker,we were to use CSIDE .Naturally all of my team are hard core Builder aficionados and boy that was a journey.I probably think we were the first org to do something with CSIDE so naturally I gave them a lot of screenshots and dumps and OT dev was very receptive.In extremely tight time constraints we developed parallel modules in 97 1 and 10 and just just sucked them up for our SRC code maintaining thing called TFS. We also learned how to work parallel on oscript but not being a code churning house our code is not that complicated so if I was working on a module somebody else was working on another as well.

So the environment prepared was

1. 971 VM with the same Oracle client and the same DB type of clone.
2. I removed Prod EFS and Prod Admin server info from clone.
3. We decided to rebuild search index as nobody had any clue whether it was good and it was eons before new search technology.
4. A base install of LL was done and the working 971 binary copy from prod was  put as an overlay.Now many starting out people do not know this technique where you can get a copy of your prod binary replete with patches et al into a playing system.BTW this is what you hear as the “Parallel Upgrade” approach .Its database connection was redefined .In very olden times when there was no differentiation between 32 and 62 bits and VM’s were not popular and cheap, it was possible to use the existing boxes and run the upgrade.This was the “Update an Instance” method.This is largely vestigial at this time and very error prone not to say you would have a real hard time going back to the working version 🙂
5. All old 971 custom modules were removed so this database had only knowledge of OT software core and optional.
6. Did my health check I said earlier like look at cats,form templates and wf maps,removed any that would hamper a upgrade.

 

At this point one can take this to a CS10.5 binary and start the upgrade.That is my next article.

 

BTW- I am not a novice when it comes to Upgrades I started using livelink software in 1999 and has worked in version 8.1.5 so I pretty much know the heartbeat of an upgrade and gets pleasure in challenges and how one methodically removes the challenges.Do not try an upgrade if you have not dry run the procedure without fail a couple of times.In most cases the upgrade can be done timely the planning can take weeks,if not months.

 

General Help Series 2-Permissions (Would the Object allow me to do this)

Permissions 

At the outset a Livelink system represents some vestiges of a File System like Unix or Dos.People having worked in that is probably familiar with the term U G O which stood for User,Group,Others.This is what you see in the std permission bit of an object on its ACL. Every object has one ,many times you set the container object right and let the inheritance trickle in.In many places if you set it right you can forget it.But if you install livelink Out of the Box and has no training or nobody to watch it over you are probably going to end up in rogue territory.This is because the user ‘Admin’ and ‘Default Group’ has very high permission bits set.Out of which the owner is a role,so if you start a folder where Admin did not willfully do the permissions set normalization and if you gave ‘Appu’ creation privileges  for folder when ‘Appu’ creates objects he becomes owner who is very powerful and so on.Now this could be argued as not a big problem until it runs amok.So what a good permission model is Owner See,See Contents or Nothing,Group See,SC or Nothing and PA,S,SC or nothing.Now you will create administrative groups something simlilar to these discussions and you will be fine.

Always rely on proper groups that you can create and maintain in a directory service (AD is very common) and have those groups synchronized in.It is quite possible that HR systems release feeds into Directory services hence when users leave your organization they will go as well.There will be no permission problems because the real administrative group has overriding permissions that the owner based approach.Simple try it and  you will be happy and 50 % of your help desk tickets will come down.

This is what one hears in OpenText engagements as “Community Modelling” well my next part will cover object truth tables as well.

Permissions bit use BitWise logic in FileSystems as well as livelink does recently a programmer told me he finds the awesome bitwise logic un understandable.I just thought wow where is the programming world heading into 🙂

There is nothing wrong in owner having high permissions so long as the administration is willing to support it and understand it.

Security Clearance and Supplemental Markings are OT’s way of securing against inheritance rules in big hush hush organizations.

Transporting Objects with Livelink

Update:10/22/2014

Me and my colleagues recently used this framework for transporting a custom subtype that we designed.We were helped by a kind OT developer All in all it saved our administration team from running XMLEXPORTS and XMLIMPORTS but the key factor is that needs to work for the transport code to work.

Update ends…

Recently OT announced a “Transport” mechanism. Users working in SAP etc are familiar with such a mechanism as changes to a baseline system are marked as configuration changes and ported to other systems.While I am oversimplifying this in a SAP deployment since so many code modules have to be configured it is impossible for an admin to remember what all has to be changed hence they decided at least 20 yrs back an enterprise program needs a transporting strategy as well.

Now lets look at how livelink gets deployed in an org. Mostly Livelink is deployed to address a document management pain/problem.Some consultants or a admin/developer who gets trained in livelink or somebody who has some documentation starts to install/configure this.assuming the person(s) have some common sense and tickets later it is working.Actually the first time install guide is very nicely written and it is mostly a “next” “next” kind of install.

Having configured it the person/persons start modelling the solution.It may involve setting up folders,categories for collecting metadata,classifications for alternative taxonomies,RM classifications ,Permissions(ACL) etc. OT and a lot of ecosystem partners take the money to sometimes set it up.You can make it complicated or simple.

Ok now that it is past the POC phase the next phase is the perhaps a TEST a more formalized place.Now comes the first shock how do I not necessarily duplicate all the work I did on a lower system.Fortunately while there was no advanced GUI or “Transport Bench” mechanism many of these building blocks can easily be transferred by XML IN/OUT operations.Smarter modules such as Classifications/RM anticipated the need for this so they did make their own transport mechanism well ahead of OT. But almost everything one could accomplish in a poor man’s methods

Now in controlled industries like Pharma etc this manual stuff was sheer heresy so companies like Global Cents made excellent products

So why the Transport now. My theory is the entry of integrations between SAP and Livelink.In fact the web services also was something my thought SAP forced OT’s hands.Many SAP customers are now getting a facelift with livelink doing a lot of backend services,pretty gui’s etc.

Here’s an interesting fictitious exchange between different players in a Integration meeting this is circa 2010,2011

SAP Basis Admin-Here we got the xECM thingy and archive server thingy going with OpenText (Notice OT in that because there is no product called that),I sthere anything else you guys need and can I close the client.

OT Consultant- Oh sure but on the next client you have to give me high access to edit the connections,change a few GUID’s(dataids) etc.

SAP Basis Admin-What the …..That means you have to do all this again

OT Consultant-Not everything just a few tweaks and pointing to the livelink admin..Hey Kishore where are we  on the next OT system. Have the ELIB items(categories and folder looking things) coming along

Livelink Admin-I did that according to documentation.

OT Consultant- How about the dataids did you run the queries I gave you for the re-mapping

Livelink Admin-Yeah everything is there

SAP Basis Admin-All the while thinking that what are these guys talking about re-configure every system.I have about 20 different clients I have to maintain.

OT Consultant-Just to prop up the SAP guy,well you know our guys in Germany are coming up with a Transport mechanism just like yours…

Well finally after a lot of back and forths the project is a success

If it is just a DMS project a lot of us have no problem using the old methods.

I am pretty sure the transport mechanism  is a GUI version of  XML Export/XMl Import.

The reason why I wrote this is recently a friend asked if he would be lucky in a project that would not have Object Importer(OI)/Object Exporter(OE) and its RM variations.

I told him this

OI/OE can be leased from OT for a price .They used to entertain that.

With XMLEXPORT/XMLIMPORT here are a few things you would run into.

1. XMLEXPORT is almost available universally but the reverse is not true universally.

. So if one exported PhysObjects the corresponding import will fail.

2. XMLEXPORT will choke on big documents and the number of nodes because of DOM parser limitations. In OE they bundle the file separately so they can get more throughput even though the code invokes a DOM parser there as well.
3. XMLEXPORT/XMLIMPORT is based on source dataids which is not going to match targets. So if you find the xml exported file and change the category id’s to match the target it will work.
4. XMLEXPORT/XMLIMPORT does not understand the hierarchy. For e.g OI will create nodes as it wants enterprise:folder1:::::::::;;;folder99. In XMLEXPORT import you would have to export and follow the reverse process. So if a child object is nested way into a path OI will handle it XML import you will have to handle it
5. Most people will use XMLEXPORT/XMLIMPORT with lapi code so they can manage a lot of things with code in java or a .net lang

Now that GUIDS are part of 10 and 10.5 I do not know how that affects anything

OI/OE is kind of a framework .So with that I was able to transport compound email,email folder and email in a project with relatively less coding.

Readers-Please tell me if the Tranport Bench from OT is good and easy to use and it resolves conflicts et al and I will delete my post.