Onboarding Agent

Helping developers reach their first "it works" moment.

Pinecone is a vector database powering AI applications. New users arrive with wildly different technical backgrounds — from interns spinning up their first index to data scientists comparing embedding models. I spent 11 weeks investigating why users churn during onboarding, then designed a chatbot-guided MVP to fix it.

The internship moved through three phases: internal insights and a console audit (weeks 1–3), participant recruitment and interviews (weeks 4–6), and design exploration with PRD development (weeks 7–11). It shipped three deliverables: a user research Notion page, a Sigma growth workbook of cohort analysis, and a chatbot PRD & MVP design.

Over 40% of users never create an index.

Joining Salesforce contacts with active accounts and Heap pageviews revealed that 40%+ of signups never create an index, signaling poor early guidance and unclear entry points. Onboarding lacked adaptive support for diverse technical backgrounds. The result: delayed time-to-value, drop-off, and lower retention.

Finding the leak with a Sigma growth workbook.

To locate where users stall, I rebuilt Pinecone's activation funnel in a Sigma workbook — joining Salesforce contacts, Heap pageviews, and BigQuery usage into one table with derived fields for title group, experience level, and engagement and revenue metrics.

Segmenting by drop-off point produced three cohorts: users who signed up but never created an index, users who created one but never activated, and multi-index power users — the success pattern worth reverse-engineering. Reading behavioral signals per account made the segments actionable:

Recruiting real users, the scrappy way.

Research began inside the building: interviews with eight internal stakeholders across Sales, Support, GTM, and Partner Engineering mapped which accounts teams consider high-value, how they segment users, and where customers stall. Outside, community emails, posts on X and Reddit, and the Cohere workshop cohort surfaced participants — 100 workshop contacts yielded exactly one screener response, a lesson in why community and sales structures matter for research pipelines. To widen the pool, I stitched Common Room community signals to Salesforce contact records, matching community members to accounts so outreach could be filtered by cohort and contacted users' onboarding behavior tracked downstream. Seven interviews across personas (interns, engineers, data scientists, PMs), 40 Decipher session replays, and a console friction audit mapped where users get stuck.

What the interviews kept repeating.

A chatbot that adapts to who's onboarding.

The goal: reduce jargon confusion and improve index creation and activation rates. Four onboarding questions tune everything the chatbot says — use case prioritizes sample apps and terminology; experience level adjusts the complexity of language and code; tech stack links relevant SDKs and model-compatibility info; and enterprise potential routes to pricing or conversion support.

Friction hotspots on the index creation page become chatbot triggers and help appears exactly where the friction audit showed users stall.

A chatbot wasn't the default answer — I weighed it against a guided product tour and static embedded docs. One-time pop-ups are easy to ignore, and static content can't react to what a user is doing. The co-pilot is both proactive (hotspot triggers) and reactive (open chat), and it keeps working after onboarding ends.

The prototype in action.

Two prompts, three personas, one eval loop.

The co-pilot runs on Claude with two entry modes: a cold-open intro message when users click the icon unprompted, and a friction-triggered system prompt that receives the hovered term plus the user's onboarding responses. The prompt classifies the user — experienced or scaling vs. new or personal-project — then adapts: performance tuning and API references for one, glossaries and getting-started guides for the other. Every recommendation must cite a doc, carry a 1–2 sentence rationale, and stay under 50 words.

Onboarding answers compress into an internal persona tag that travels with the session, so tone and assumptions are set before the widget even loads:

The design also had a real implementation path: the spec mapped onto Inkeep's embedded-chat SDK — its intro message for the cold open, per-hotspot example questions, and session-level context prompts carrying the onboarding answers — with the widget's ikp- component classes restyled to feel native to the console. Engineering could scope the MVP without building a widget from scratch.

Prompt variants weren't judged by vibes: I iterated three prompt versions — from a bare instruction set to explicit Pinenut/Sapling/Sequoia persona mapping — and ran them through Anthropic's prompt evaluation tool against three synthetic test users (a no-code explorer, a Python hobbyist bringing embeddings, a production AI-agent builder), each defined by a hovered friction point plus onboarding answers. Outputs were compared for grounding, doc relevance, and tone fit — following Anthropic's eval-design guidance: task-specific test cases and automated, rubric-driven LLM grading over slow hand-grading. The doc mapping behind each response was vetted with Pinecone's docs team before handoff.

Signals that grow with the user.

More data types can be captured over time via Heap and pre-populated into the chatbot for support along the PLG lifecycle.

Measuring success, then expanding.

Primary metrics: A/B testing for index creation uplift (+10% from the current 70%), qualified tooltip hovers via Heap (median dwell time, ignoring sub-300ms jitters, capped at 5s), and chatbot CTR — ≥50% of the tested cohort views it, ≥30% click a suggestion, ≥2 messages per engaged session.

Secondary: time-to-task completion, prompt effectiveness via Anthropic's prompt evaluation tool, and qualitative thumbs up/down feedback. The MVP ships read-only — it never modifies user vectors or indexes, and prompts are shaped only by non-PII metadata.

If the MVP proves out, the chatbot extends into activation and scaling flows: onboarding survey + Heap behavioral signals → cohort tagging → dynamically updated support that also feeds GTM insights.

Learning to navigate ambiguity.

The biggest lesson from Pinecone wasn't the experience of working at a frontier AI infrastructure company. It was learning how to navigate and grow in ambiguity.