A/B Testing and Benchmarking Benchmarking compares recommendation strategies while keeping assignment and attribution consistent. Use Froomle-managed assignment when Froomle owns the split, or customer-managed assignment when the customer’s experimentation system owns it. Both use the same measurement contract. Assignment ownership and rendering ownership are separate decisions. First decide who assigns the visitor to a branch, then decide which application component renders and tracks the content for that branch. Agree the number and exact names of benchmark groups with your Froomle account team before implementation. Use those names consistently in configuration, response-driven rendering, and reporting. Whether an outbound request or event contains user_group depends on assignment ownership as defined below. This guide uses control and treatment as the branch names. Unless different group names are agreed, the SDK configuration maps the control branch to customer and the treatment branch to froomle. Choose the Benchmark Shape and Assignment Owner First decide whether the benchmark compares Froomle models or uses existing control content. For control-vs-treatment, choose the assignment owner: Froomle-managed model vs. model. Froomle assigns the visitor and every branch renders returned recommendations. This is the simplest setup and uses the normal recommendation integration; control content and frontend control-item identity are not required. Froomle-managed control vs. treatment. Use this when existing editorial or internal content is the control and Froomle assigns the visitor. The frontend must explicitly enable froomle-managed benchmark behavior and give every control item a stable item ID and the correct type so both branches can be measured. Customer-managed control vs. treatment. Use this when the customer’s experimentation system assigns the visitor. The frontend must explicitly enable customer-managed behavior, supply the externally resolved current group, and give every control item a stable item ID and the correct type. For a control-vs-treatment benchmark implemented with the frontend SDK, benchmark configuration and control-item identity are two separate requirements. You need both: Benchmark mode: configure froomle-managed or customer-managed. For customer-managed assignment, also provide the current group resolved by the external experimentation system. Stable control identity: provide an ID and item type for every displayed control item. Declarative DOM integrations should prefer direct data-froomle-id / data-froomle-item-type, and may use the SDK’s selector fallback when existing CMS markup cannot be changed. React integrations provide getId(…) and, when needed, getItemType(…) adapters. Setting only an ID selector does not enable A/B testing. Setting only benchmark mode does not make unidentified control content measurable. Control Identity Preference For SDK-managed declarative DOM placements, use identity sources in this order: Direct DOM identity (preferred): put data-froomle-id and, when needed, data-froomle-item-type on the control item itself. Selector fallback: when existing CMS markup cannot be changed, configure a selector and source attribute from which the SDK copies the control identity. If neither approach can identify the rendered control item, the application must own its rendering and recommendation-event tracking instead. Scenario 1: Froomle-Managed Split Froomle assigns visitors to the configured groups. The assignment is stable for the device, including anonymous visitors who later log in. Option A: Full A/B Test (Model vs. Model) Froomle compares two or more recommendation strategies. The response contains the items selected for the visitor’s assigned strategy, and the application renders the returned result. Option B: Control vs. Treatment Existing editorial or internal content is the control; Froomle recommendations are the treatment. The response user_group is the branch authority. Resolved branch Returned items Expected rendering Control Non-empty or empty Render control content. Do not select a branch from item count alone. Treatment Non-empty Render the returned Froomle items. Treatment Empty Render the agreed empty or fallback state for the Froomle branch. For Froomle-managed assignment: Do not pass user_group to select the request branch. Preserve the returned request_id and resolved user_group through rendering. Attribute impressions and clicks to the response that produced the rendered branch, but do not echo the returned user_group in those events. Froomle assigns the version and group server-side. Scenario 2: Customer-Managed Split Use customer-managed assignment when the customer’s experimentation platform should decide the visitor’s branch. For each request: Resolve the visitor’s branch in the customer experimentation system. Pass the agreed user_group for the control branch. Omit user_group for the treatment branch so normal Froomle version resolution applies. Render content according to the resolved branch, not according to whether returned items happen to be empty. Track impressions and clicks for the rendered branch with the same attribution decision. Control recommendation events carry the agreed control user_group. Treatment events omit user_group and rely on the returned request_id for attribution. User Group Overview Concern Froomle-managed Customer-managed Who assigns the branch? Froomle Customer experimentation system Pass user_group in the request? No Only for control; omit it for treatment Rendering authority Returned user_group Customer decision, represented by the agreed request contract Control events Omit user_group; preserve request attribution and let the Events API assign it Include the agreed control user_group Treatment events Omit user_group; attribute through request_id Omit user_group and attribute through request_id This matrix is implementation guidance for manual/custom API calls and bare programmatic SDK rendering. For SDK-owned declarative DOM and React placements, the SDK applies it automatically: keep rendering through the supported SDK primitives and do not duplicate branch-attribution logic in application code. Event Requirements A benchmark is valid only when shown content is measured for every branch. Send an impression for every displayed recommendation or control item included in the benchmark. Send a click_on_recommendation when a displayed recommendation or control item is clicked. Keep item, placement, request, and branch attribution aligned with what the visitor actually saw. Attribution field Shared contract action_item Stable ID of the item that was shown or clicked. action_item_type Item type used by the synchronized catalog. list_name Placement/list being benchmarked. request_id ID returned by the recommendation response. Never invent or hard-code it. user_group Include only the externally assigned control group in a customer-managed split. Omit it for customer-managed treatment and for every Froomle-managed branch. When both branches follow this attribution contract, Froomle dashboards can report their performance side by side using the same measurement pipeline. Frontend SDK Control-vs-Treatment Checklist Before enabling traffic for a control-vs-treatment benchmark, verify all of the following: The placement explicitly configures froomle-managed or customer-managed benchmark mode. An ID selector alone is not benchmark configuration. Customer-managed assignment supplies the externally resolved current group; it is not hard-coded to one group for every visitor. Every control item has one stable ID and the correct item type. For declarative DOM, prefer direct data-froomle-id and data-froomle-item-type; use selector fallback only when existing CMS markup cannot be changed. The control branch keeps control content and the treatment branch renders returned Froomle content according to the resolved assignment. Both branches produce recommendation impressions and clicks with the same list_name and returned request_id contract. window.FroomleFrontendSdkRuntime.diagnostics.benchmark shows the expected mode and resolved branch. For selector fallback, diagnostics.benchmark.controlIdentity.lastResults confirms whether each placement used an explicit ID, a selector-derived ID, or failed as missing or ambiguous. Consent Benchmark assignment must not override consent rules. Anonymous requests remain anonymous, and tracking events are sent only at a consent level that permits them. Keep only the routing information needed to resolve the agreed benchmark branch. Implement the benchmark Method Implementation guide Frontend SDK Benchmarking and A/B Testing with the SDK Manual / custom integration Manual recommendation benchmarking and Manual event integration Related Topics Performance & Metrics Event model and tracking contract Recommendation request model