Splunk Observability Suite Onboarding
Senior User Researcher
Challenge:
Splunk's unified UI team sought to streamline the onboarding process and consolidate the experiences of recently acquired products into a cohesive suite-wide experience. The goal was to accelerate time-to-value (TTV) during onboarding, especially for users leveraging the free trial, and determine the optimal adoption path for maximizing value from the Observability Cloud Suite.
Research Questions:
-
What is the quickest path to value during onboarding?
-
What are the primary pain points during onboarding?
-
What is the ideal product installation order?
-
Are there quick wins that demonstrate value and drive adoption?
Methodology:
The research employed a mixed-methods approach, including:
-
Preliminary workshops with stakeholders and sales team to establish a baseline.
-
In-depth semi-structured interviews with 12 active users about their onboarding experience.
-
Remote observation of 8 new users completing onboarding tasks, including their use of documentation.
Key Insights:
-
APM Paradox: While application performance monitoring (APM) is central to observability, the onboarding process almost always begins with basic server monitoring (IMM). Setting up APM to truly get the most value out of it requires an expert developer to integrate OpenTelemetry (OTel) into your environment, which isn’t very common.
-
Docs as a Proxy: Clarity in documentation is viewed as a proxy for the ease of use of the product. We need to provide proper, concise documentation within the product as it is needed.
-
F1 Kit Car Analogy: Splunk's Observability potential is vast, but the complexity of setup and configuration demands a skilled team and strategic implementation, whereas competitors like Datadog offer a more user-friendly experience.
Findings:
-
What is the quickest path to value during onboarding?
-
Infrastructure is the quickest path towards value, but the game changing value we promise comes with the additional integration of APM which takes time and collaboration amongst multiple teams and multiple programing languages. This is almost impossible to accomplish during the week long free trial.
-
-
What are the primary pain points during onboarding?
-
The specific technical difficulties customers run into during onboarding were dependent on their amount of experience and unique tech stack but the most common theme we heard was a lack of clarity in documentation. Another common issue was wrangling other teammates and convincing engineers to alter their code at the source.
-
-
What is the ideal product installation order?
-
First, Infrastructure Monitoring is the base.
-
Second, Application Performance Monitoring provides unparalleled insight but is dependent on Infrastructure Monitoring.
-
Third, Log Observer can effectively reduce MTTR building on IMM and APM.
-
Last, Synthetic Monitoring, Real User Monitoring and On-call are useful but dependent on the customers needs and tech stack.
-
-
Are there quick wins that demonstrate value and drive adoption?
-
The quickest wins are obtained through Synthetics and RUM but may not apply to everyone and do not create enough lasting value on their own.
-
Outcomes:
-
Affinity Journey Maps: Detailed user journeys that highlight pain points and opportunities for improvement. There was a consistent trend of rising initial excitement when learning about the potential power of the tools in the suite, leading to a quick dip of disappointment when faced with the complex realities of customized implementation.
-
Product-Market Misalignment: Identified discrepancies between product capabilities and market needs, emphasizing the need for a more accessible onboarding experience. Marketing a direct trial with immature self-service infrastructure was a mistake and it was harming our brand.
-
Documentation Integration: Incorporated documentation review and observation into future user research studies, revealing low-hanging usability issues.
Impact:
This research project played a pivotal role in understanding and addressing the challenges of onboarding users to Splunk's Observability Cloud Suite. By identifying pain points, clarifying the product-market fit, and driving strategic changes, the research significantly contributed to improving the user experience.
-
Strategic Shift: The research led to the de-prioritization of self-service trials, reducing churn and improving customer satisfaction.
-
Targeted Go-To-Market: The focus shifted towards existing Splunk Cloud customers, leveraging our existing technical sales teams to guide customers during their Observability Suite trials complex installatiton.
For a more detailed case study please contact me at Wolff.RyanM@gmail.com