How to find the right test tool for Okta, Auth0, and other SSO solutions
If you purchased a single sign-on (SSO) product, how do you know it is working properly? It sounds like a simple question, but answering it is not that simple. Setting up automated connections will require an understanding of the authentication protocols they use. You will also need to know how your different applications use these protocols (on-premises and SaaS) to encode them correctly in the SSO portal.
Although the major single sign-on providers support hundreds of apps, some of them, especially your own apps, may not already be in their catalogs. This means that you might have the non-trivial effort of writing your own custom login scripts. Some single sign-on providers have tools or consultants (available as part of the initial purchase or at an additional cost) to help you.
Wouldn’t it be nice if you could run an automated test tool to find out where you slipped or where your SSO software is failing? This can be useful in three scenarios:
- When your app vendors make small changes to their login screens, those changes can trigger your automation routines, causing your SSO login to fail. The major SSO providers claim they have ways of staying in the know, but if they don’t, your users will usually let you know before you can fix the issue.
- When you deploy a new app, whether it’s a public SaaS app or something you’ve developed in-house and want to make sure your SSO connection is working properly.
- When a malicious actor infiltrates your business. (More on that later.)
To cover all of these scenarios, you have two basic choices: The first is to use the testing tools written by your SSO provider. The problem is, finding these tools isn’t easy. Many vendors view these tools as a side development project rather than something they offer with their consumer product lines.
The second choice is to consider a third-party testing tool. A growing collection of these tools can help you determine if you have correctly configured the many variables, crypto tokens, and strategies. The caveat is that they only work with a few of the SSO products, at least so far. While the lack of a testing tool shouldn’t influence your SSO purchase decision, it’s good to know if one is available for your eventual implementation.
Another option is to use a Cloud Access Security Broker (CASB) tool. These do other things besides automating the connection to your SaaS applications, like adding another layer of security.
Testing tools from major SSO vendors
When you first launch your SSO implementation, you usually have a small group of user testing to make sure that it works as expected for a small sample of your applications. Some SSO providers may create a test environment to resolve bugs, such as provisioning and deprovisioning users, and identifying unknown (or at least unknown to the IT organization) applications that will require attention. particular.
This is where things get interesting. If this is your first exposure to Security Assertion Markup Language (SAML), OpenID Connect, and Open Authorization (OAuth) protocols, you’ll want help with how your SSO provider implements them. All sellers have extensive (and almost illegible) documentation. This is where these testing tools can come in handy, as they don’t require you to study protocol syntax but provide a more user-friendly interface. To get a better idea of what to expect, you can read this blog from Testim, which is a general supplier of software automation testing. They cover the typical SSO testing setup steps.
Two SSO providers have details on SSO testing which are a good place to start, especially if you don’t know the difference between a SAML service provider and an identity provider. You don’t need to be a customer to use most of these tools from CyberArk (which bought the SSO line of products from Centrify) and NetIQ (renamed by Microfocus to CyberRes).
CyberArk SAML test tool series are available in different languages (PHP, Python, Ruby, Java and .NET). A nice bonus is that they are all available on GitHub. They also have a open source SAML tracing tool, so that you can debug and troubleshoot your SAML message traffic from your browser. NetIQ public SAML library series include a larger collection of programming languages.
You should probably have a separate test instance for your entire web application environment so that you can experiment with single sign-on without damaging your production systems. Three suppliers brought them together:
- NetIQ has its Access Manager OAuth Playground, where you can try things out with their identity management product.
- Ping Identity has its OAuth Playground also for this purpose, where you can experiment with OAuth and OpenID Connect scripts without damaging your production systems. Ping to other tools, such as a SAML decoder where you can examine XML request commands.
- Okta has something similar with its integration network. It has pre-built connectors to thousands of apps and much more: tools for additional security, analytics, and common SaaS apps. There are also plentiful documentation on how to create your own integrations, as well as “wizards” on building SAML and OpenID Connect integrations. While that’s a lot to review, at least there’s a good chance they’ve come across most of the apps your users are running.
If you use other SSO products, you will have to resort to a patchwork of tools:
- Auth0 offers help troubleshoot SAML implementations, but manual steps to resolve various error messages can be tedious. You can select a special debugging mode to get a series of more detailed error messages. They also built a service that maintains the changes made to the services once they are created. Even if you don’t use Auth0, this is a useful reference if you want to learn more about SAML troubleshooting.
- Matt Fuller offers useful debugging tips for users of Amazon single sign-on and other identity management implementations for its web services.
Third-party SSO testing tools
The problem with the SSO provider’s toolset is that you have no way to automate testing across your entire application infrastructure. This is where third party products come in, such as Testim mentioned earlier. Testim is a generalist software test automation vendor, of which there are dozens, if not hundreds (like Silk and Selenium, for example). While Testim doesn’t offer any SSO-specific tools, you can whip up a way to test your SSO implementation if you have specialists on your staff (or outsourced) who favor a particular automation program. Other software testing companies cover SSO implementations:
Careful works with Azure Active Directory and OneLogin. An Okta version is in beta. It requires built-in Canny widgets to pass the SSO token. Pricing starts at $ 50 per month.
Dorothy of Elastic works only with Okta and requires Elastic Security. Pricing starts at $ 16 per month.
SSOScan is an open source vulnerability scanner for Facebook SSO only.
Walrus works with Okta, Auth0, IBM, ForgeRock and NetIQ. It is based on its general purpose automated testing tool. The price is not disclosed.
Many of these testing tools come from the general test automation space, which means these vendors can test a lot more than your SSO applications. This is both good and bad news. Good because it shows that automation vendors are finally paying attention to SSO troubleshooting. Bad because the number of SSO clients is still a small subset of each vendor’s user base, which means it can be difficult to find a competent support person.
In fact, finding support is a challenge. None of these suppliers would answer our questions. Another downside is that getting the documentation on features related to single sign-on will also require some effort. (In Walrus’s case, getting their price proved impossible.)
The tools offer common functionality:
- Automatic fault detection. If a login script fails, you can identify it before your users call in to complain.
- Quick creation of new test scripts. Once you have mastered the ins and outs of the tool, you can generate new scripts in a matter of minutes. Writing scripts should also be easier than debugging SAML or JSON code error messages.
- Software vulnerability testing. (I cover this below.)
If you are using Okta’s SSO module, you can try most of these automation tools and see which one you like best. If you’re using Auth0 (which Okta acquired earlier this year but maintains as an independent product), Ping, or whatever, you have more limited options until vendors expand their support.
Security vulnerability testing
When putting together an SSO implementation, you will likely have some issues to ensure that a bad actor does not slip into your infrastructure. In an extreme example, a hacker successfully poisoned crypto tokens to elevate Active Directory privileges.
Other circumstances are more common, such as:
- Can you easily determine if users who are no longer employed have been successfully removed from your connections?
- Can you search for unnecessary administrator users, simple passwords, or poorly configured network zones?
- Can you analyze a provider’s SSO log and if so how is it done? (Elastic’s Dorothy Tool, for example, uses Fleet for precisely this purpose.)
You might think that general purpose vulnerability scanners can perform some of these tasks, but again, don’t necessarily focus on the sanctity of your SSO system. Enter SSOScan, which is the only general purpose SSO vulnerability scanner I could find, made even more limited because it will only work with Facebook SSO connections. Because it’s open source, you can take a look at how it’s built and maybe modify it to work with your SSO implementation. It can be more effort than it’s worth if you don’t use and don’t care about Facebook integrations.
Testing your SSO setup can be frustrating. There are some new third-party tools that can help you, especially if you are running Okta’s SSO environment. Other single sign-on providers have been slow to develop better automated methods. If you’re using Okta, Ping, or Auth0 SSO, start with their tools. If you have anything else, start with the CyberArk or NetIQ code libraries. Then add your layer of automation with the tool you are most familiar with.
Copyright © 2021 IDG Communications, Inc.