HCL Connections & Kerberos Authentication Protocol Issue

After implementing Kerberos Authentication protocol for HCL Connections, as described in the official documentation (HCL Connections and IBM WebSphere documentation) and restarting the whole environment, the “synchronization status” of the Nodes in the IBM WebSphere ISC Console appeared to be “unknown”. All the HCL Connections Applications were running, there were no errors in GUI and the SSO was working without any issues. From the logs of the node agents we could see that the synchronization with the deployment manager was taking place and completing successfully, only in the “SystemOut.log” of the WepSphere deployment manager I found the following error message:

000015c Krb5WSSecurit E SECJ9314E: An unexpected exception occurred when trying to run initSecContext() method : GSSException: org.ietf.jgss.GSSException, major code: 11, minor code: 41


major string: General failure, unspecified at GSSAPI level
minor string: Kerberos error formatting credential for delegation: 41
at com.ibm.security.jgss.i18n.I18NException.throwGSSException(Unknown Source)
at com.ibm.security.jgss.mech.krb5.g.a(Unknown Source)
at com.ibm.security.jgss.mech.krb5.g.a(Unknown Source)
at com.ibm.security.jgss.mech.krb5.g.initSecContext(Unknown Source)
at com.ibm.security.jgss.GSSContextImpl.initSecContext(Unknown Source)
at com.ibm.security.jgss.GSSContextImpl.initSecContext(Unknown Source)
at com.ibm.ISecurityLocalObjectTokenBaseImpl.Krb5WSSecurityContextImpl$1.run(Krb5WSSecurityContextImpl.java:508)
at java.security.AccessController.doPrivileged(AccessController.java:770)
at javax.security.auth.Subject.doAs(Subject.java:570)
at com.ibm.ISecurityLocalObjectTokenBaseImpl.Krb5WSSecurityContextImpl.initSecContext(Krb5WSSecurityContextImpl.java:273)
at com.ibm.ISecurityLocalObjectTokenBaseImpl.Krb5WSSecurityContextImpl.initSecContext(Krb5WSSecurityContextImpl.java:157)
at com.ibm.ws.management.connector.soap.SOAPConnectorClient.processInternal(SOAPConnectorClient.java:1285)
at com.ibm.ws.management.connector.soap.SOAPConnectorClient.getSecurityHeader(SOAPConnectorClient.java:1095)
at com.ibm.ws.management.connector.soap.SOAPConnectorClient.invokeTemplateOnce(SOAPConnectorClient.java:783)
at com.ibm.ws.management.connector.soap.SOAPConnectorClient.invokeTemplate(SOAPConnectorClient.java:697)
at com.ibm.ws.management.connector.soap.SOAPConnectorClient.invokeTemplate(SOAPConnectorClient.java:687)
at com.ibm.ws.management.connector.soap.SOAPConnectorClient.queryNames(SOAPConnectorClient.java:599)
at com.ibm.ws.management.connector.soap.SOAPConnectorClient.invoke(SOAPConnectorClient.java:524)
at com.sun.proxy.$Proxy40.queryNames(Unknown Source)
at com.ibm.ws.management.AdminClientImpl.queryNames(AdminClientImpl.java:108)
at com.ibm.ws.management.AdminServiceImpl.queryNames(AdminServiceImpl.java:679)
at com.ibm.ws.management.connector.AdminServiceDelegator.queryNames(AdminServiceDelegator.java:113)
at sun.reflect.GeneratedMethodAccessor88.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:508)
at com.ibm.ws.management.connector.soap.SOAPConnector.invoke(SOAPConnector.java:503)
at com.ibm.ws.management.connector.soap.SOAPConnector.service(SOAPConnector.java:335)
at com.ibm.ws.management.connector.soap.SOAPConnection.handleRequest(SOAPConnection.java:65)
at com.ibm.ws.http.HttpConnection.readAndHandleRequest(HttpConnection.java:733)
at com.ibm.ws.http.HttpConnection.run(HttpConnection.java:522)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1892)

After a quick Google search I found that this boils down to a problem with a hostname defined or spelled in upper case letters. As described in the following article:

https://www.ibm.com/support/pages/apar/PM66457

This article also mentions a custom variable which could be set to remedy this issue.

But before trying the custom variable out, I wanted to make sure that this was really the issue. I checked the DNS Server and all SPN records, I could not find any issues there. The output of the “hostname” and “hostnamectl” commands (the whole environment is running on RHEL), were all returning the hostname written in (all) lowercase characters. Only when looking into the “hosts” file of the deployment manager I could find the issue, there was a “127.0.1.1” entry stylized in “CamelCase”. I replaced the upper case letters in the “hosts” file with lower case letters and as soon as I have done that, all the nodes in the ISC console had the status “synchronized” and the error mentioned above did not appear anymore. The custom variable was not needed.

I hope this saves you some time! πŸ™‚

HCL Traveler Database Move – From DB2 to MS SQL – Made Easy

One of my customers wanted to migrate the HCL Traveler Database from IBM DB2 to Microsoft SQL. The customer is using Microsoft SQL for all other applications, which is set up with redundancy and high availability in mind, so this was a sensible choice, opposed to running a single instance of DB2.

EDIT: A migration of the Traveler data, inside the enterprise database, is currently not supported, and I do not know of anyone who has done it successfully. Nevertheless, the move towards another enterprise database was desired by the customer as long as there was no action needed on behalf of the users (entering their credentials again for example), this means that the ActiveSync devices will need to resynchronize their data after the move.

EDIT: There are also no tools to migrate the HCL Traveler data from one enterprise database to another. Although the database is not so complicated, and it may be possible to write your own tool for “low-level” SQL Data migration towards another enterprise database (this is just a thought of mine and I have not tried this out). We decided to continue with a simple move towards another enterprise database, meaning that the end user devices will have to “re-sync” most of their data again, this could be an issue in a larger environment, so when in doubt start with a PoC environment and plan accordingly.

First thing to do is to create a new SQL Database and open up the needed Firewall ports to access it. In our case it was the port 1433, but you may need to check your database server settings, as this may differ depending on the configuration and/or technology used. After that you should create a new service user solely for the traveler connection, this user should have appropriate rights to the database/instance. Consult the official documentation for the appropriate SQL Server permissions.

Before proceeding with the next steps, make sure to test the connection to the SQL Database, using the service user you created earlier. I usually to this using a “UDL” file, as described in the separate post.

After the database is created and the connection is tested, I strongly advise configuring at least two test devices, using the same settings and the same application as the majority of the users are using at the customer site. This is useful during the migration, as you can see the impact on the users during the process and verify that everything is running smoothly.

One important thing to keep in mind is that all the settings in “LotusTraveler.nsf” Database (HCL Traveler Administration Database, accessed only via HTTP/HTTPS) will be lost after migrating to the new Database. This is not a big deal, as there are only a handful of settings involved, take note of them before moving to the new database and do not forget to apply them once you have moved to the new enterprise database.

Before configuring HCL Traveler to use the new database on MS SQL, you will need to download the appropriate JDBC driver for the MS SQL. Just download them using the URL in the official documentation, and you are good to go. After downloading the driver package, shutdown the Traveler Servers, extract and copy the JDBC driver to the “<Domino_Program_Directory>\Traveler\lib\” directory. Make sure to delete the DB2 JDBC drivers which are not needed anymore, I have not done this at first, and it caused a problem with the connection to the new database.

At this point, the devices using HCL Traveler will not be able to sync, due to the Traveler servers being offline. Now you can reconfigure the Database connection on the Traveler servers. Just run the “travelerUtil” tool, using the command appropriate for your environment, as described in the official article. There is no need to run the “db remove” command, just make sure to check the database related settings in the “notes.ini” after running the “travelerUtil” tool, these need to be correct prior to the start of the HCL Traveler Server.

HCL Traveler “travelerUtil” command.

After running the command, run the “travelerUtil db show” and “travelerUtil db check” command and verify the settings. This is a very important to be done before starting the HCL Traveler Domino task.

Before starting the HCL Traveler server for the first time, I would start only the HCL Domino Server without starting the Traveler server task and start it manually afterwards, just so we could make sure that we see all Traveler related messages on the console. You can do this by removing the Traveler task from the list of tasks in the “notes.ini” (“ServerTask” parameter).

After starting the Traveler server for the first time, do not forget to set the settings in the “LotusTraveler.nsf” which you noted previously. This should be done only while the HCL Traveler Server (Traveler task) is running, only then will the changes be written in the new enterprise database.

As soon as the Traveler server is started, all user devices will start with the full re-sync, at this customer site we needed about 1 hour for about 150 Devices. The performance varies from environment to environment. Important to note is that we have done this during the weekend, and we did not get any user complaints nor the users had to enter their credentials again.

A word of warning, before making any configuration changes, make sure that you have created a consistent backup. While doing this procedure, the most important thing is testing along the way, as every change will have immediate user impact. Also make sure to consult the official documentation and check if all the requirements for the HCL Traveler are fulfilled. I especially advise you to read the section about the HCL Traveler High Availability Pool. It is also very important to be running the latest version of the HCL Traveler Servers, as I am sure that with some older versions of Traveler, this process is not so smooth and the initial Sync of the devices using ActiveSync will take a longer time to complete. In this environment, devices do not need to be “approved” by the administrator to connect to the HCL Traveler Servers via ActiveSync, if that is the case in your environment, you should test the impact of this feature in a PoC environment before moving towards a different database in your production environment.

Many thanks to Detlev Poettgen for explaining this procedure to me, make sure to keep an eye to his blog, as his posts are very helpful! Also, big thumbs up for the very helpful HCL Support team.

Let me know if you would like me to cover any part of this procedure in greater detail or if you would like to see this as a Session at a conference!

HCL SafeLinx 1.1.1 Version available!

Since Monday there is a new version of HCL SafeLinx available. Make sure to install it, as it contains some very important fixes. Some SafeLinx functions will not work properly without it. For example, the HCL Nomad client connectivity will not be possible without this fix.

SafeLinx 1.1.1 provides fixes for the following issues:

SAFE-540: Failover not working properly for Verse mail. When configured in failover or round robin mode connect failures would not trigger a failover to other Domino servers.
SAFE-568: Redirect port not working when multiple HTTP services are defined. If multiple HTTP services are configured to redirect from port 80 to port 443, only the first service would get redirected.
SAFE-569: Replace EULA with master copy.
SAFE-573: Secure LDAP Bind authentication broken. TLS start returns -12, LDAP_NOT_SUPPORTED. Added option to allow untrusted certificates.
SAFE-585: Add cache-control to login page to cache images.
SAFE-592: Dead-Lock in restart after crash when multiple HTTP services are defined on the same port.
SAFE-598: MEM_BAD_POINTER on exit of wg_acct.
SAFE-600: Nomad mobile-only configuration fails to generate userConfig.json.
SAFE-601: Crash in network congestion recovery. Simultaneous tear down may cause race condition.
SAFE-602: Migration fails if KDB password contains shell special characters. Issues with migration when multiple HTTP services are defined, only the first service gets migrated properly.
SAFE-626: Secure Administrator configuration not reading the PKCS12 password from wgated.conf.
SAFE-627: Add Strict-Transport-Security and generic header token add function.

Official Release Notes:

I’m proud to be an HCL Ambassador for 2021!

First of all, a massive THANK YOU to everyone who voted for me. I am proud to be selected as HCL Ambassador for 2021, it is a pleasure and a great honor to be in this group of people.

HCL Ambassador Class of 2021

This will give me an earlier access to the beta codes, product news and better connections inside of HCL. Not to mention that it will motivate me a ton. πŸ™‚

In turn, I will be able to write blog posts and get even more involved in this great community!

For anyone who is not familiar with the program, this is how HCL defines the role of HCL Ambassador:

HCL Ambassador is a distinction that HCL awards select members of the community that are both experts in their field and are passionate about sharing their HCL knowledge with others.Β 
​
HCL Ambassadors are exactly that, ambassadors. Importantly they are not employees, but their commitment to sharing their expertise has a huge impact on the HCL community. Whether they are blogging, writing books, speaking, running workshops, creating tutorials and classes, offering support in forums, or organizing and contributing to local events – they help make HCL’s mission of making technology play nice, possible.

Last but not least, I want to congratulate to all HCL Ambassadors!

HCL SafeLinx 1.1.1 & HTTP Strict-Transport-Security

After a SafeLinx Deployment I wanted to set the HTTP Strict-Transport-Security header, but there was nothing in the documentation about it, and I also could not find any option regarding this in the SafeLinx Administrator client settings. So I opened a Support case.

According to the support you can use the command line on the HCL SafeLinx server to set the HTTP Strict-Transport-Security header as well as any other token header. Example of the command:

chwg -s ibm-wlHttpService -l http-serviceX -a hcl-strictTransSec=TRUE -a hcl-httptokens=”MyToken: my value” -a hcl-httptokens=”MyTOken2: my value2″

As of the upcoming HCL SafeLinx 1.1.1 release, which is coming next week, there is going to be a form to set it in the HCL SafeLinx Administrator client. Also, the HTTP Strict-Transport-Security header will be enabled by default in this release. Yay! πŸ™‚

Domino, Designer and Notes v12 Roadmap

During the first day of the virtual HCL Factory Tour we were able to see the roadmap for Domino, Designer and Notes v12. HCL continues to innovate and sets the bar high in regard to product quality. The new release of Domino, version 12, is coming in Q2 of 2021.

HCL Domino Roadmap

HCL Domino v12 release is going to focus on the following three key points.

Basically it focuses on Volt, new possibilities in regard to infrastructure and better overall experience by delivering a new web based client. Rather than that, at a glance the v12 will deliver the following new features:

HCL Domino v12 Features

The version 12 of Domino will be supported on more Linux platforms than the previous release.

New Linux Platforms in HCL Domino v12

AstraLinux support seems to be very important on the Russian market. As mentioned earlier in the course of the HCL Digital Week, the backup process will be redesigned so it could be implemented easily by any third party solution. All major cloud vendors will also be supported.

New backup and cloud possibilities

From the security aspect, the usage of Let’s Encrypt certificates as well as 2FA will be possible out of the box.

With Domino v12, we will get a new client. I am especially excited about that. This may solve many issues that we currently have in terms of support with Windows Terminal clients for example.

HCL Domino v12 will also deliver some new capabilities for HCL Verse on-Premises, currently the focus lies on the following features.

HCL Verse on-Premises – Top priorities

After the delivery of HCL Nomad web client, HCL aims to deliver an all integrated user experience for the desktop clients. I can’t wait to get this, the users would benefit greatly from a UI which connects the HCL collaboration products.

Integrated user experience

HCL Notes client will also get updated and get some long awaited features. The UI (including workspace) will be improved to increase efficiency. Alternate “From” will be introduced, so we can switch on the fly from which mail account an e-mail should be sent or which signature should be used. The Performance of the client will be improved as well. A very welcome addition will also be the inclusion of Language packs as part of the basic installation so there will be no need to install them separately. The Notes Client will also get a new branding and very own icon! πŸ™‚

As expected, the new release of the Notes Designer will focus on mobile devices.

HCL Notes Designer v12

HCL continues to rapidly improve and develop the Domino platform by focusing on its strengths. I am sure that the HCL is going in the right direction!

HCL Sametime Premium announced – It’s a stunner!

As a part of HCL Digital Week, Luis and Gini presented the HCL Sametime Premium. It is intuitive, easy to use and frankly everything what we want from a modern video conferencing solution and more. The emphasis is put on cost savings, according to HCL you can save, for ten thousand users, over a million us dollars annually! In the following I will write a brief summary of the session, along with a few thoughts of my own.

According to HCL, when deploying video conferencing software, the customers are met with the following problems:

These problems are tackled with the HCL Sametime Premium.

As mentioned, a lot of emphasis is set on cost savings. The amount of money that HCL Sametime Premium can help you save, just blew me away, amazing! Take a look at the following slide.

Cost savings with HCL Sametime Premium

I like it that we still decide where we can deploy the solution, not like at some other software vendors…

HCL Sametime Premium and Sametime v11.5 offers a lot of new functionality, the following slide does not represent the full set of features, just the most important functionality at a glance.

HCL Sametime Premium – Features at glance

Luis also showed us a very cool demo with Sametime Premium, mostly Meetings, in action. You can create new Sametime Meetings in HCL Notes, or in HCL Nomad Web client, no problem.

HCL Nomad Web client

All functions are intuitive and easy to access, like screen sharing.

Screen Sharing

YouTube video sharing, as well as streaming on YouTube is available out of the box! I personally really like this feature.

HCL Sametime Premium integration with YouTube

The transition to mobile devices is easy and seamless.

By the way, you can install the mobile app via QR-Code, you just need to enter your username and password and you are ready to go. This is a welcome addition, I always wanted to spare the user of typing the server name and other needed settings.

HCL Sametime App installation via QR-Code

This is how a meeting can be created via web client.

HCL Sametime Meetings web client

Another cool thing is that the meeting recording is available just seconds after ending the recording of a meeting.

Downloading or sharing a meeting recording

Luis also showed us that you can deploy HCL Sametime Premium, with all features, in a day! Including setting up AWS, or doing some other preparation work on your platform of choice. To help us gain knowledge on how to do this, HCL will publish some new Whitepapers.

HCL Sametime whitepapers

HCL Sametime 11.5 including Meetings is available for download today, including a calculator tool to help you leverage just how much money you can save with HCL Sametime Premium.

Get Started!

All in all Sametime Premium and/or Meetings is a very welcome addition, there are so many customers waiting for Sametime Meetings. I hope that we are going to use HCL Sametime in many of our future projects. In the end always remember:

Be like Carla! πŸ™‚

HCL Domino v12 Preview – HCL Digital Week

Today, as a part of the HCL Digital Week, we had an opportunity to take a glimpse into the future and the v12 version of HCL Domino. In the following I will write a brief summary of the session, along with a few thoughts of my own.

In terms of upgrading to the v12 release, we got to see a live demo of the update from v9 to v12 of Domino. It took only about 5 to 10 Minutes to successfully upgrade the Domino server to the version 12. Business as usual. πŸ™‚

It looks like the emphasis of the new version will not be on the traditional Notes client, as of now, it does not look like the HCL Notes 12 will get a major overhaul from the previous version. Although, we got to see the “Type Ahead Search” feature, which is a welcome addition!

On the other hand, we will get a new web based client, HCL Nomad Web client. Which gets me extremely excited because this client has the full functionality of the traditional Notes client. HCL Nomad Web client will run on all most popular Browsers (Firefox, Chrome, Safari…) and will be supported on Windows, Linux and macOS!

HCL Nomad Web Client – Supported Platforms

Here are some screenshots from the new HCL Nomad Web Client:

HCL Nomad Web client can be installed by opening a URL in a web browser, the whole installation takes about a minute (although this is dependable on your network bandwidth). The access, to the installation page, can be secured via two-factor authentication (2FA).

HCL Nomad Web client installation

This brings us to other cool features which will come with HCL Domino v12, FaceID and other biometric means of authentication will be supported in HCL Nomad mobile client.

HCL Nomad Mobile client

It will be possible to integrate HCL Domino Applications in Microsoft Teams, as long as the Domino Applications can be opened in web.

HCL Verse on-Premises will be fully supported in a web browser on mobile devices.

On the other hand, on desktop, HCL is working on a fully integrated user experience, this is something I can’t wait to get. Things like Verse on-Premises and Sametime Meetings integration. It will be possible to access a Sametime Meeting from a mobile device, by scanning a QR-Code from Verse on-Premises Client.

There was a lot of talk today about cloud and containerization and HCL Domino v12 session was no exception, Domino v12 will run on most major cloud platforms, HCL also guarantees that your backup solution will support Domino v12. The whole backup process will be reinvented so it could be easily supported by all backup software vendors.

Cloud and Backup

Which brings us to the “Cloud Native” journey. When installing HCL Domino v12, a “one click install” will be possible by using a JSON configuration file. In terms of ease of deployment and automation, this means a great deal.

HCL Domino’s Cloud Native Journey

A few new security features will also be implemented. As mentioned, 2FA and biometric authentication on mobile devices will be supported out of the box as well as the whole process of obtaining and using the Let’s Encrypt SSL certificates. Active Directory Password Sync will also be available in the next release, one password for AD, HCL Notes and Domino Web Access, finally! πŸ™‚

HCL Notes Designer v12 is going to allow easier development of responsive applications, here we have a clear focus on developing and modernizing applications for mobile devices.

HCL Domino Volt is going to be enhanced even more, a more simplified web administration page will be delivered as well as connectors for third party solutions.

HCL Domino Volt and Domino v12

Domino v12 will be available in 2021 in Q2 until then we can participate in HCL Nomad Web Beta!

HCL Nomad Web Beta

HCL Domino v12 will bring a lot of new highly requested features, I can’t wait for it to come, the orientation is clearly on mobile and cloud, as well as building a secure and highly functional backend for the rapid low- and pro-code development.

Nominations are open for HCL Ambassadors!

It is that nice time of the year again, the nominations for HCL Ambassadors are open! If you feel that any individual has helped you personally or contributed to the HCL Community, outside their defined area of work, please do take a minute to nominate that person. You can do it under the following URL:

https://www.hclambassadors.com/volt-apps/anon/org/app/17958248-1415-4d09-81bf-f084ebfb6d7c/launch/index.html?form=F_NewFormDashboard1

The HCL Ambassador program allows the nominee to access a lot of betas, information, help, support and some cool assets, which in turn is going to benefit you.

Visit the official HCL Ambassador Website to find out more about the program.

Please do not feel obliged to nominate me by any means, but if you think that I deserve it, you can paste the following information to the nomination form:

Firstname: Milan
Lastname: Matejic
E-mail: milan.matejic@axians.at
Country: Austria
Company: Axians ICT Austria GmbH
Job Title: IT Consultant
Current HCL Ambassador: No
HCL Brand: Digital Solutions
Blog url: https://milanmatejic.wordpress.com
Linkedin: https://www.linkedin.com/in/milan-matejic-45536511a/
Twitter: @Milan_Matejic90
Facebook: https://www.facebook.com/milan.s.matejic
Products: HCL Domino/Notes, HCL Connections, HCL Sametime, HCL Safelinx

Summary:

I am giving my best to blog in my spare time about HCL Products, Achivements and Fixes I have self found.
I was speaking at Engage 2020, I held a Session “Implementing Certificate Based Authentication for HCL Traveler Access”. More on that under the following URL:

https://milanmatejic.wordpress.com/2020/03/30/engageug-implementing-certificate-based-authentication-for-hcl-traveler-access/

Inside my company I advertise about HCL Collaboration Products and help my colleagues inside and outside the company build Know-How with these Products, by having remote Sessions with them.
I managed to convince our Customers to continue to use HCL Domino and HCL Connections, and not migrate to other platform.

In the last five years I haven’t missed a single ICS Event in Vienna and I gave my best to take as many as possible colleagues and customers to it.
I was strongly advertising for HCL Domino v10 and v11, blogging frequently about it and spreading the news to our customers. I also encouraged and helped Colleagues and Customers to
Submit Ideas on HCL Domino and HCL Connections Ideas Blog.

Highlights:

1) Speaking at Engage 2020

https://milanmatejic.wordpress.com/2020/03/30/engageug-implementing-certificate-based-authentication-for-hcl-traveler-access/

2) Blogging & pushing HCL News on social media (Twitter, Facebook, LinkedIn, Xing,…)

https://milanmatejic.wordpress.com/

3) Member and contributor of Enthusiasts Groups regarding HCL Collaboration Software (Domino, Sametime Connections)

-) OpenNTF Slack Channel (Domino, Sametime, Connections)

-) Facebook groups

https://www.facebook.com/groups/LotusDominoAdmins/

https://www.facebook.com/groups/87320249925/

https://www.facebook.com/groups/101414713319478/

And if you do nominate me, please let me know! πŸ˜‰

HCL Sametime – Update site and SAML enabled webserver

If you plan to manage your HCL Sametime clients via Expeditor managed settings framework and automatically updating their preferences via “managed-settings.xml” file, make sure that the file(s) are placed on a web server, in that way, so these files can be accessed without any form of authentication.

A SAML enabled server may look like a good idea, but at least in my tests, I could not get it to work with HCL Sametime Embedded client.