A/B Testing and Benchmarking Benchmarking compares recommendation strategies while keeping assignment and attribution consistent. Froomle-managed assignment is recommended because it places every branch on the same assignment and measurement foundation. Assignment ownership and rendering ownership are separate decisions. First decide who assigns the visitor to a branch, then decide which application component renders and tracks the content for that branch. Agree the number and exact names of benchmark groups with your Froomle account team before implementation. The same names must be used consistently in requests, rendering decisions, events, and reporting. Scenario 1: Froomle-Managed Split (Recommended) Froomle assigns visitors to the configured groups. The assignment is stable for the device, including anonymous visitors who later log in. Option A: Full A/B Test (Model vs. Model) Froomle compares two or more recommendation strategies. The response contains the items selected for the visitor’s assigned strategy, and the application renders the returned result. Option B: Froomle vs. Internal Baseline Froomle compares its recommendation strategy with customer/control content. The response user_group is the branch authority: Resolved branch Returned items Expected rendering Customer/control Non-empty or empty Render customer/control content. Do not select a branch from item count alone. Treatment/Froomle Non-empty Render the returned Froomle items. Treatment/Froomle Empty Render the agreed empty or fallback state for the Froomle branch. For Froomle-managed assignment: Do not pass user_group to select the request branch. Preserve the returned request_id and resolved user_group through rendering. Attribute impressions and clicks to the response that produced the rendered branch. Scenario 2: Customer-Managed Split Use customer-managed assignment only when an external experimentation platform must decide the visitor’s branch. For each request: Resolve the visitor’s branch in the customer experimentation system. Pass the agreed user_group for the customer/control branch. Omit user_group for the treatment/Froomle branch so normal Froomle version resolution applies. Render content according to the resolved branch, not according to whether returned items happen to be empty. Track impressions and clicks for the rendered branch with the same attribution decision. Customer/control recommendation events carry the agreed control user_group. Treatment/Froomle events omit user_group and rely on the returned request_id for attribution. User Group Overview Concern Froomle-managed Customer-managed Who assigns the branch? Froomle Customer experimentation system Pass user_group in the request? No Only for customer/control; omit it for treatment/Froomle Rendering authority Returned user_group Customer decision, represented by the agreed request contract Customer/control events Preserve the resolved branch attribution from the response Include the agreed control user_group Treatment/Froomle events Attribute through request_id Omit user_group and attribute through request_id Event Requirements A benchmark is valid only when shown content is measured for every branch. Send an impression for every displayed recommendation or control item included in the benchmark. Send a click_on_recommendation when a displayed recommendation or control item is clicked. Keep item, placement, request, and branch attribution aligned with what the visitor actually saw. Attribution field Shared contract action_item Stable ID of the item that was shown or clicked. action_item_type Item type used by the synchronized catalog. list_name Placement/list being benchmarked. request_id ID returned by the recommendation response. Never invent or hard-code it. user_group Include the customer/control group when the customer-managed contract requires it; omit it for treatment/Froomle events that use request attribution. When both branches follow this attribution contract, Froomle dashboards can report their performance side by side using the same measurement pipeline. Consent Benchmark assignment must not override consent rules. Anonymous requests remain anonymous, and tracking events are sent only at a consent level that permits them. Keep only the routing information needed to resolve the agreed benchmark branch. Implement the benchmark Method Implementation guide Frontend SDK Benchmarking and A/B Testing with the SDK Manual / custom integration Manual recommendation benchmarking and Manual event integration Related Topics Performance & Metrics Event model and tracking contract Recommendation request model