# Running a Warehouse Native POC

## What Statsig Warehouse Native does

**Statsig Warehouse Native** enables customers with existing metric logs to analyze that data, and optionally bring previous assignment data or offline experiments into the platform. Statsig WHN has two types of experiments:

<ol>
<li>**Assign and Analyze**: You can run an experiment on **web/mobile/app** and use Statsig’s SDKs to assign (bucket or randomize) users, and then analyze results.</li>
<li>**Analyze**: You can run an experiment elsewhere (***your own SDK, email, direct mail, sms, ivr, etc.***) and use Statsig to analyze that data and calculate experiment results. Statsig reads this assignment data from your warehouse in [this format](https://docs.statsig.com/statsig-warehouse-native/configuration/assignment-sources#example-data); Statsig calls these [Assignment Sources](https://docs.statsig.com/statsig-warehouse-native/configuration/assignment-sources).</li>
</ol>

If you have a pre-existing experiment in your warehouse, start with an **Analyze** experiment. An Analyze experiment is an effective way (less than 1 day) to establish a connection between Statsig and your warehouse, and to experience [experiment results](https://docs.statsig.com/experiments/interpreting-results/read-results) in the Statsig console. After that, run an **Assign and Analyze** experiment using Statsig's SDKs (typically an A/A test). The A/A **Assign and Analyze** experiment lets you test Statsig's SDKs and implementation process with your engineering and product team.

## Steps to running an effective proof of concept with Warehouse Native

{% figure %}
![Warehouse Native POC workflow diagram](https://docs.statsig.com/images/statsig-warehouse-native/guides/running_a_poc/723b6c60-7151-4716-8970-0e52c391991d.png)
{% /figure %}

Follow these steps when planning your Warehouse Native implementation:

1. **Define your experiment(s) and metrics for validation** - A proof of concept determines if Statsig fits your experimentation needs; running an experiment is the most direct path for evaluation. **Responsible Party**: Typically a product or engineering lead.
   - Plan to run 1-2 production level experiments to validate. Past experiments, A/A tests or upcoming projects or product changes are great opportunities to implement a Statsig experiment!
     - Identify your hypothesis and [metrics](https://docs.statsig.com/statsig-warehouse-native/configuration/metrics) to validate this hypothesis. Statsig joins these metrics with unit assignment/exposure data and runs them through the stats engine.
     - If your team plans on running **analysis only**, identify the user **assignment** data that Statsig joins with the metric data.
       - The analysis-only approach can yield results in as little as **30 minutes**, assuming data is readily available for ingestion.
     - If your team plans on using the **Assign and Analyze** experimentation option, identify **where** the experiment runs. Typically, **web-based** experiments are easier to evaluate, but Statsig also has SDK support for server and mobile SDKs.
       - **Note**: It’s important the implementing team understands how the SDKs operate before executing a proof of concept. The [client](https://docs.statsig.com/client/introduction) and [server](https://docs.statsig.com/server/introduction) docs can help orient your team.
       - A typical evaluation takes **2-4 weeks** to account for experiment design, implementation, time to gather data, and analysis. To ensure a successful POC, [have a well scoped plan](https://docs.statsig.com/statsig-warehouse-native/guides/running_a_poc#2-phase-0-scope--prepare-your-poc) and include the right teams to assist.
   - Read [experimentation best practices](https://statsig.com/blog/product-experimentation-best-practices) to learn how to succeed.

2. **Connect the warehouse** - To query data and operate within your warehouse, you need to allocate resources and connect to Statsig. You may choose to use an existing prod database or create a separate cluster specifically for experimentation (if you don’t already have one).
   - Statsig requires a role and the following access:
     - Read access to metric and exposure data
     - Write access so Statsig can write results and exposures back to the warehouse
     - Access to run jobs and query data
       - Find more guidance on [connecting with your specific warehouse vendor here](https://docs.statsig.com/statsig-warehouse-native/guides/connect).
   - Review the [data pipeline overview](https://docs.statsig.com/statsig-warehouse-native/analysis-tools/pipeline-overview) to understand how data flows for warehouse native jobs.

3. **Connect metric sources and define metrics** - After you connect the data warehouse, you can begin defining metric and assignment sources (if applicable) in Statsig. Statsig expects specific schemas to correctly map the data to its pipelines:

   {% figure %}
   ![Schema requirements diagram for metric sources](https://docs.statsig.com/images/statsig-warehouse-native/guides/running_a_poc/4fa8c5e2-c19f-4ef6-be70-f31b61170e7c.png)
   {% /figure %}

   Beyond these columns, the schema is flexible and can accept additional columns. You can use **Metadata** to filter metrics and perform more granular analysis.
   - Review guides for creating a [metric](https://docs.statsig.com/statsig-warehouse-native/configuration/metric-sources) and [assignment](https://docs.statsig.com/statsig-warehouse-native/configuration/assignment-sources) source
   - Follow the [data best practices](https://docs.statsig.com/statsig-warehouse-native/guides/best-practices) to ensure your queries are running efficiently.

   {% callout type="info" %}
   This section is important to review and can prevent unnecessary infrastructure costs!
   {% /callout %}

   After you connect metric sources, configure [metrics](https://docs.statsig.com/statsig-warehouse-native/configuration/metrics) to perform various aggregations (e.g., Sum, Mean, Count, Unique Users) that represent what you’re trying to measure in your experiments.
   - [Supported metric types](https://docs.statsig.com/statsig-warehouse-native/configuration/metrics#metric-types) and ways to [configure them](https://docs.statsig.com/statsig-warehouse-native/configuration/metrics)
   - Use [cohort metrics](https://docs.statsig.com/statsig-warehouse-native/features/cohort-metrics) to measure impact during a certain time frame per user.

4. **Create and roll out an experiment** - In step 1, you defined the planned experiment(s) and the metrics used to validate them. With metrics and assignment sources configured, you can now create the experiment(s). Refer to the [detailed guide for experiment setup](https://docs.statsig.com/statsig-warehouse-native/guides/experiments), and consider these things as you complete this step:
   - Create your hypothesis and select the **experiment (assignment) source**.
     - If using the SDK for assignment, the SDK itself serves as the assignment source.
   - You can use [Custom IDs](https://docs.statsig.com/guides/experiment-on-custom-id-types) but must first configure them; for example: `device_id`, `vehicleId`.
   - Refer to [advanced settings](https://docs.statsig.com/statsig-warehouse-native/features/configure-an-experiment#advanced-settings) for the many ways you can configure your experiment.
   - As you roll out your experiment, you can [monitor the status with health checks](https://docs.statsig.com/statsig-warehouse-native/features/monitor-an-experiment) and get a readout of live exposures as they come through the SDK.
   - If you’re hoping to quickly validate the platform, you can create and run a [quick A/A test](https://docs.statsig.com/statsig-warehouse-native/guides/aatest).

5. **Read results** - After the experiment runs successfully, read the results and ensure everything looks reasonable. Did the results validate your hypothesis, or are they surprising? Are the results easy to interpret and navigate for the teams involved? Refer to the [section on Pulse](https://docs.statsig.com/statsig-warehouse-native/guides/pulse) to understand the high-level analytics capabilities. A few things to note:
   - You can slice results further using the [*Explore* tab](https://docs.statsig.com/statsig-warehouse-native/guides/pulse#explore), which lets you break down results by specific user and event properties.
   - You can configure Statsig to forward exposure and metric data to your warehouse.
   - The [*Health Checks* (diagnostics) tab](https://docs.statsig.com/statsig-warehouse-native/guides/pulse#health-checks) surfaces the SQL used to generate results so you can validate any analysis performed on your systems.

6. **Finalize evaluation and next steps:** Ultimately, a POC validates a set of evaluation criteria that determines whether Statsig is a good fit for your team’s workflows. The following graphic provides high-level guidance on what to look for during your evaluation phase.

   {% figure %}
   ![POC evaluation checklist graphic](https://docs.statsig.com/images/statsig-warehouse-native/guides/running_a_poc/c5f86db5-6bf9-4a5e-9709-ede36384b3ca.png)
   {% /figure %}

To move forward, convert your POC environment to a production-level implementation. Refer to the [production guide](https://docs.statsig.com/statsig-warehouse-native/guides/production) for details.

Statsig offers support through the [community Slack channel](https://statsig.com/slack).