Building a native AB Testing tool with GA4
This is a high-level summary of steps necessary for basic AB testing. This set-up will track test views and conversions for each test group. Other variables such as time spent on page or views/session are not included, though could easily be integrated by the end-user in GA.
There are three components necessary for custom AB testing with Google Analytics:
Assignment. First, we randomly assigning each site visitor to one side of the test. Typically there are two: “A” and “B”, though this method could be modified to track more than two test cases.
Rendering. Next, we render the page content conditionally based on the current user’s test group. Ideally this is done server side, but client side is also possible.
Tracking. Finally there is conversion tracking. We can do this through Google Analytics, by sending events for both test views and clicks or conversions for each test case. For example, button A viewed and button A clicked, button B viewed and button B clicked. This data will give you a conversion percentage, and will also ensure that the test has been configured correctly by indicating that the test groups are being assigned evenly and randomly.
Step 1: Random assignment
Users are assigned randomly to a group by generating a random number (e.g. 0 for A, 1 for B) on page load. That assignment is stored in a cookie in the user’s browser. Implementing this server side is recommended.
In order to run multiple tests simultaneously, it is advised to store the test assignment in a JSON object as a key value pair, where the key is the test in question, and the value is the user’s assignment.
To avoid reassigning the same user to a different test group, the key for that test should be checked within the cookie before the cookie is (re)generated.
It’s also recommended to encrypt the JSON object inside the cookie to avoid a user manually changing their assignment, and also more importantly to obfuscate the tests currently being run.
One downside to this approach is the same person may see both sides of a test if they use a different browser or device during the test duration, also users must have consented to having cookies stored in their browser. An alternative would be a database table with columns for user (username if logged in, IP otherwise), test, and assignment.
Step 2: Rendering the content
Once a user is assigned a test group, we know what content we need to show them. There are many ways of doing this depending on the website platform, technology and developer access.
Two options are outlined below:
Option 1: Conditional rendering
Ideally, the page should only render the content for the test case the current user is assigned to. This can be done if the contents of the cookie can be checked before the page is rendered, and the DOM content can be modified accordingly. This works great for Wordpress sites. We’ve also built this on a few React websites.
Option 2: Conditional display
This option is best for conditions where server side rendering or client side DOM modification is less accessible, such as sites built with page builder interfaces, or hosted by third parties such as Shopify. In this approach, both “A” and “B” content is rendered simultaneously for all users, and the content for the version the user is not in is retroactively hidden via a CSS rule. That rule is applied via client-side javascript.
This option is less desirable due to the possibility of page “jumping” during rendering (could affect CLS core web vitals) and the unwieldiness for the content editor of a page with duplicate content for each test case. There is also the possibility that such an approach could affect SEO, as content from both versions could be indexed by Google (unless conditional rendering is applied on Googlebot user agents)
Step 3: Google events
The last step for AB testing setup is gathering data for each test case. In this example we are using Google Tag Manager and Google Analytics.
First, ensure that your GTM tag is installed in both the head and body of your website.
Instructions for doing so can be found here:
https://support.google.com/tagmanager/answer/6103696
Next, create Triggers in Tag Manager for each test group and action being tracked. For example, testing 2 variations of a web form would require 4 Triggers:
Form A Viewed
Form A Submitted
Form B Viewed
Form B Submitted
For the “viewed” triggers, the simplest approach is to use the “CSS Selector” selection method.
In this implementation, each test case should have a unique selector, #form-a and #form-b for example. The Trigger Type should be set to “Element Visibility”, and the trigger should be set to fire once per page. If the content in question is rendered to the DOM on the client side (i.e. after page load) the “Observe DOM changes” option should be checked.
“Submitted” triggers can be triggered in the same way as the “viewed” trigger. In this case, the CSS selector can be configured to look for any content that is rendered after a successful submission (taking care to differentiate between test groups). Alternatively the Trigger Type can be set to “Click - All Elements” and be set to fire when a link or button with a specified CSS selector is clicked. There are several other trigger types available, and alternatives may be more suitable for the particular test case. It is recommended to use the “preview” tool to make sure the triggers are firing correctly.
Once triggers are defined, it is necessary to send events to Google Analytics. This is done by creating Tags in GTM for each trigger. In the “Tags” section of GTM, create a new tag. The tag type should be set to Google Analytics: GA4 event, and then select the trigger you want to send as an event.
Once these are set up, your custom events should begin to appear in Analytics under Lifecycle->Engagement->Events. Note that it may take a day or so for your new events to begin to appear in Google Analytics, though the data should begin to be captured right away. From this point forward you can use the GA dashboard to build segments and create reports on the events, as you would with other events. You can pull in any value you wish (sessions, conversions, etc.) and see it split across the two test versions.
