What is an Experiment?

An experiment is a list of modifications. Given the created list, you want to know what and how each modification from the list impacts your sampled users. Based on the experiment's outcome, you can decide whether the experiment has been deemed to be successful or not and respectively, apply the changes to your platform.

Creating a Client-side experiment

Client-side experiments allow customers to provide changes to the website and analyze the changes on the “fly” with the built-in editor.

An experiment can be created from the Experiment builder object by clicking on the Create experiment button. You will be first asked to provide a descriptive name for the experiment.

Afterward, you will be prompted to select the Experiment type: either Client-side or Server-side. Select the Client-side option.

As a final step, you should enter the URL of the website you want to run the experiment on. It can be a homepage or a specific page of your platform. Note that entering the specific URL does not limit the experiment running on that very page.

Experiment editor

Once the experiment has been created, you will be redirected to the stated URL with an Experiment editor extension.

List of changes

You can access the list of modified elements and the applied modifications using the “Changes” button. Each change in the list shows the modified elements and the changes made in the modification.

NOTE: You can hover over a modification to highlight the edited element. You can also click on a modification to change it.

Personalized modifications

When a modification is personalized (it contains Liquid), it will be marked in the modification list. Personalized modifications load slower than modifications that are not personalized. This also affects any modifications that are applied after the personalized modifications. If possible, sort your modification list in such a way that all (or as many as possible) non-personalized modifications are placed before the personalized modifications.

Experiment variants

Experiment variants serve as a tool to compare different modifications of the same text, picture, structure, functionality, etc. while comparing it to a control group. You can modify the variants of the experiment using the variant selector (labeled with the number 1 in the previous screenshot). It is possible to duplicate, rename, and delete a variant.

NOTE: There's always a variant called a Control group. This is how your website looks without the changes to any other variant. This variant cannot be changed or deleted.

Selector

Selector is a CSS selector of the current modification (or the currently selected element(s) on the page). Whenever you are hovering over an element or selecting an element on the webpage, the editor automatically generates a unique selector for that element. If you hold the Shift key, you can select multiple elements by clicking, as could be seen in the picture above. The editor then generates a selector that matches all the selected elements.

After selecting an element(s) by clicking on it, you can still edit the auto-generated CSS selector by clicking on the pencil-shaped icon in the top bar. For example, your page may have many different pictures, and adjusting all of them would take too much time; select one picture and rewrite the selector in order to select more (or all) elements. You can even write an entirely custom CSS selector to apply modifications to if you know CSS.

Undo/redo

With Undo/redo you can undo or redo your changes to the experiment/website.

Library

From the Library tab you can access the list of created Popups that can be inserted into the experiment. Additionally, you can also create HTML elements and add them to your experiment.

Preview

By clicking on the “Preview” button, it will generate a link to your modified website that you can share with others.

Desktop/mobile view

By clicking on the Desktop/mobile switch, you will be able to preview your experiment URL in either Desktop or Mobile configurations.

Modifications list

The menu pops up when you click on an element on the website. From the list, you can edit the HTML element.

Change

Use this option to change the element visually. You can edit the dimensions, colors, fonts, spacing of the element, etc. You can enter the colors manually (using the hex or RGB color format) or use the color picker. When changing the font, make sure it is loaded onto the website. And finally, the sizes need a number and a unit (for example 43px or 12em). The format you should use is indicated in the fields as a placeholder. This modification type also allows you to change an image on the website and a link's URL address. In order to do that, you have to first select an image or a link.

If you are advanced, you can switch to the Code tab and modify the HTML of the selected element. You can use Liquid syntax here to personalize the content for the user.

Insert

This will open a menu with your Popups and HTML Blocks saved in the Library. In both cases, you can either pick an already existing template or create a new one using the code editor.

Move

Moving an element has two parts. Before selecting an anchor point on the website, you need to select whether you want to place your currently selected element before or after that anchor point. You can also put it inside that anchor point as the first or last child. Before and After are self-explanatory and also the most-used options — the new content will be placed right before or after the selected element. Using these two options places the new content outside the selected anchor element which means that the new content will not inherit the style of the selected element. If you put the element inside as the first or last child, it will inherit the style of the anchor element.

Move

Change your current selection to the parent or a child of the currently selected element. You have to understand a little bit of HTML and its structure to use this effectively. Selecting a parent element means going one level up within the website structure. Each element can also contain multiple children, and you can select one of them to move down within the website structure.

Creating a Server-side experiment

Server-side experiments let you use Intempt’s A/B testing while fully controlling how the experiments are rendered on your site. Variations of your custom Server-side experiments are JSON files, so each call results in a JSON that you can use to render an element on your site.

After providing a descriptive name for the experiment, you will be prompted to select the Experiment type: either Client-side or Server-side. Select the Server-side option.

Once the experiment has been named, you will be directed to the Experiment itself where you need to add and configure the experiment variants. By clicking on the Add new variant you will be presented with the variants templates. You can either choose one of the suggested templates or create one by yourself.

Enter a JSON payload variation for a selected section in your site.

If you click on the Variables option you can enter the values for the added payload.

Variants

For both the created Client-sideU and Server-side experiments the Variants* tab displays the list of available variants. From this tab, you can edit, add, and delete the experiment variants.

Goals

The automatic AB test optimization and the automatically generated evaluation report are based on this setting.

Target audience

Define your custom audience that will see this experiment. You can use our standard customer and event filters.

Display

Set until when you want to keep showing the experiment for each eligible customer.

Target devices

Set on which devices the experiment should be run.

Show on

Choose based on which URLs to show the experiment.

Schedule

You can choose to display the experiment instantly or set a specific time period in which the experiment is shown.

Analytics

Every experiment has a built-in evaluation dashboard. This dashboard displays common metrics and analyses.

The Control group represents the group of subjects that are set aside and do not receive the content with modifications.
Reach measures the number and percentage of customers that viewed the experience
Conversion specifies the percentage of users that completed the goal set in the settings section (a goal can be either event-based or segment-based).
Lift-over control shows the difference in improving conversion rate in the variants compared to the control group (customers that received unchanged web page experience).