Appearance
Why your CASB is lying to you about Shadow IT
The Cloud Access Security Broker category has been a quiet success story for the security industry. Gartner's most recent Magic Quadrant for Security Service Edge (SSE) shows the major vendors converging on a comparable feature set, and most large enterprises now run at least one CASB in some configuration. The category's central premise — that you can give security teams visibility into the SaaS applications employees use without IT approval — has become a baseline expectation, not a differentiator.
Over the past four months, CipherGuard tested how well that premise actually holds. We built a controlled corporate environment, populated it with 312 real SaaS apps across 27 functional categories, and ran four widely-deployed CASB products against it. We then compared what each vendor told us was being used against ground truth.
The discovery rates ranged from 38% at the low end to 71% at the high end. That gap is large enough to matter, and the shape of the gap — which apps each vendor missed — was predictable from how each vendor builds its catalog.
How the test was constructed
We provisioned a 50-user simulated tenant on Microsoft Entra ID with realistic role distributions (engineering, sales, marketing, finance, operations, HR). Each persona was given a scripted browsing schedule that, over four weeks, touched 312 apps drawn from three sources:
- The Productiv State of SaaS 2025 top-500 list, filtered to apps that have a public web interface and a recognisable domain pattern.
- The BetterCloud SaaS Trends Report longtail (apps adopted by fewer than 1% of enterprises).
- A hand-curated list of 47 deliberately obscure regional apps (Korean accounting SaaS, German HR portals, Latin American expense tools) plus 12 apps that are technically web-hosted but very rarely catalogued (e.g., the Bombardier flight-planning portal, several semiconductor design vendors' customer portals).
All traffic was routed through each CASB's traffic-collection layer in turn (a mix of agent-based, reverse-proxy, and API-only modes, configured per vendor recommendation). Cleartext sampling was checked against a packet-capture baseline. Where vendors offered both proxy-mode and API-mode discovery, we ran both and took the union.
The headline numbers
| Vendor | Apps discovered | Discovery rate | False-positive rate |
|---|---|---|---|
| Vendor A (full-suite SSE) | 222 | 71.2% | 4.1% |
| Vendor B (CASB-led platform) | 198 | 63.5% | 6.3% |
| Vendor C (network-overlay) | 173 | 55.4% | 8.0% |
| Vendor D (endpoint-led) | 119 | 38.1% | 11.6% |
We have not named the vendors in this report. The point is not to elevate or punish any single product; the point is the predictable structural reasons for the gap. Vendors who wish to know how they scored, and against which 312 apps, may contact us directly.
False positives — apps the CASB claimed were in use but were not — averaged 7.5% across the four. The most common cause was treating CDN domains (cdn.<saas>.com) as separate apps from their parent product. The second most common was conflating different products from the same vendor.
The gap is structural, not technical
What predicted whether a given app would be discovered by a given vendor turned out to be almost entirely a function of how that vendor builds its app catalog. There are three dominant approaches:
1. Vendor-submitted catalogs
Two of the four CASBs derive most of their catalog from vendor-submitted data: SaaS providers fill out an integration form, supply domain patterns, OAuth scopes and traffic fingerprints, and get a green tick in the CASB's directory in exchange for what is effectively a co-marketing relationship. These catalogs are very accurate for the apps that bother to submit. They are nearly empty for everything else.
In our sample, the 47 deliberately-obscure regional apps were discovered at a rate of 6.4% by these vendors. The 12 industry-specific portals were discovered at 8.3%. By contrast, the top-100 mainstream apps were discovered at 94%.
2. Crawled / scraped catalogs
One vendor builds its catalog by continuously scraping the web, looking for distinctive traffic patterns (specific JavaScript files, Open Graph metadata, OAuth provider strings) and inferring the existence of a SaaS app. This approach scales better to the long tail but produces more false positives (the 11.6% false-positive rate above is this vendor) and is fragile to apps that change their frontend stack.
The crawled-catalog vendor caught 34% of our regional apps and 42% of the industry portals — the highest of the four — but missed 11% of the mainstream top-100, mostly because the apps in question had recently overhauled their domain structure and the crawler had not re-indexed.
3. Endpoint-led discovery
The fourth approach uses an endpoint agent to enumerate browser bookmarks, recently-visited URLs, and OAuth consents directly from the user's machine. In theory this should be the most accurate; in practice it suffers from two structural problems. First, it under-counts apps used in incognito sessions or on unmanaged endpoints (BYOD, contractors, the marketing team's MacBooks). Second, the agent's URL classifier is itself dependent on a small catalog — usually a stripped-down version of the vendor-submitted one above — so once it sees a URL, it still has to recognise it.
The endpoint-led product was the worst performer in our test for a reason: its agent was deployed on roughly 70% of the simulated endpoints (matching real-world deployment rates we see at clients), and within that 70% it had to rely on a vendor-submitted catalog that itself missed most of the long tail.
What this means in practice
The defensive implications fall into three buckets, in order of how immediately actionable they are.
Stop treating CASB discovery numbers as ground truth
If your CASB tells you that you have 280 SaaS apps in use, the actual number is likely between 400 and 700, and almost all of the unknowns will be in the long tail — which is where data-loss risk is least understood. The mainstream apps (the Slacks, the Salesforces, the Box-and-Dropboxes) are well-instrumented because regulators and vendors made them well-instrumented. The unknowns are the small expense tools, the niche design SaaS, the regional vendors your sales team in Seoul is using.
A practical interim mitigation: cross-reference your CASB's discovery output with your DNS resolver logs and your SSO IDP's full OAuth consent history. The union will give you a substantially higher number, and the deltas are the ones worth investigating.
Treat region and industry as catalog axes
When you procure a CASB, the question to ask is not "how many apps are in your catalog?" — every vendor will quote a six-figure number that includes every WordPress plugin in existence. The right question is "how many of my apps are in your catalog?" The way to answer it is to hand the vendor a list of 30–50 apps you know are in use in your organisation, including at least 10 that are obscure to your industry, and check the hit rate before you sign.
Don't outsource the long tail
The category of apps a CASB will reliably not catch is the category most likely to host the most sensitive data, because those apps are usually adopted by small high-leverage teams (finance, M&A, executive admins) precisely because they are not on IT's radar. The only tool that will reliably surface these is human conversation — quarterly "what tools is your team actually using?" conversations with department leads.
This is not a satisfying answer for a security programme that wants to automate everything. It is the answer that matches reality.
What we would test next
We expect to revisit this study in twelve months with a few changes:
- Add a fifth vendor in the browser-isolation-plus-discovery category, which has matured significantly since our planning began.
- Expand the regional sample to include more Japanese, Brazilian and Indian apps; the European and Asian coverage in our current corpus is uneven.
- Test against AI-tool sprawl specifically. Several of the largest SaaS-policy gaps in 2026 are around model providers and prompt-management tools, and discovery in this category was poor across all four vendors (we excluded it from the headline number because the category is moving too fast to measure cleanly in a four-month window).
If you are a CASB vendor, a SaaS-discovery vendor, or an enterprise security team with a different methodology to compare, we would like to hear from you at [email protected].
Disclosure. This work was funded entirely out of CipherGuard's editorial budget. No vendor paid for inclusion. No vendor had advance sight of the results. The simulated environment, the persona scripts, and the app-list will be made available to the four tested vendors on request.