Study is an AB testing library designed to be clear, minimal, and flexible. It works in both the server and browser with the use of driver-based persistence layers.
You can download the compiled javascript directly here
Storage Drivers: localStorage, cookies, memory, or build your own
Well documented, tested, and proven in high production environments
Lightweight, weighing in at ~ 3.8kb.
Not tested on animals
Installing
# Via NPMnpm i studyjs --save# Via Bowerbower i studyjs --save# Via Yarnyarn add studyjs
Developing
npm install # Install dependenciesnpm build # Build the babel'd versionnpm lint # Run lintingnpm test # Run tests
Usage
<script src="study.js"></script><script> // Set up our test API const test = new Study({ store: Study.stores.local }); // Define a test test.define({ name: 'new-homepage', buckets: { control: { weight: 0.6 }, versionA: { weight: 0.2 }, versionB: { weight: 0.2 }, } }); // Bucket the user test.assign(); // Fetch assignments at a later point const info
API
Study(config)
const study = new Study({ debug: true, store: Study.stores.local});
This creates a new test API used to defined tests, assign buckets, and retrieve information.
Returns: Object
Name
Type
Description
Default
debug
Boolean
Set to true to enable logging of additional information
false
store
Object
An object with get/set properties that will accept information to help persist and retrieve tests
Study.stores.local
study.define(testData)
// Create your test APIconst study = new Study();// Define a teststudy.define({ name: 'MyTestName', buckets: { variantA: { weight: 0.5 }, variantB: { weight: 0.5 }, },});
This function defines the tests to be assigned to used during bucket assignment. This function accepts an object with two keys, name and buckets. Alternatively, you may pass an array of similar objects to define multiple tests at once.
The name value is the name of your test. The keys within bucket are your bucket names. Each bucket value is an object containing an object with an optional key weight that defaults to 1.
The percent chance a bucket is chosen for any given user is determined by the buckets weight divided by the total amount of all weights provided for an individual test. If you have three buckets with a weight of 2, 2/6 == 0.33 which means each bucket has a weight of 33%. There is no max for the total weights allowed.
Returns: null
Name
Type
Description
Default
data
Object/Array
An object/array of objects containing test and bucket information
null
study.assign(testName, bucketName)
const study = new Study();study.define({ name: 'new-homepage', buckets: { variantA: { weight: 0.5 }, variantB: { weight: 0.5 }, }});// Assign buckets from all tests to the user...study.assign();// or assign bucket from the specified test...study.assign('new-homepage');// or specify the bucket from the specified test...study.assign('new-homepage', 'variantB');// or remove the bucketing assignment from the specified test.study.assign('new-homepage', null);
Calling the assign method will assign a bucket for the provided tests to a user and persist them to the store. If a user has already been bucketed, they will not be rebucketed unless a bucketName is explicitly provided.
If no arguments are provided, all tests will have a bucket assigned to the user. If the first argument provided is a test name, it will attempt to assign a bucket for that test to a user. If a bucketValue is provided, it will set that user to the specified bucket. If the bucketValue is null, it will remove that users assignment to the bucket.
Returns: null
Name
Type
Description
Default
testName (optional)
String
The name of the test to assign a bucket to
null
bucketName (optional)
String
The name of the bucket to assign to a user
null
study.definitions()
const study = new Study();study.define({ name: 'new-homepage', buckets: { variantA: { weight: 0.5 }, variantB: { weight: 0.5 }, }});// Retrieve all of the provided testsconst tests = study.definitions();
This provides the user with all of the tests available.
The returned information will be an array if multiple tests were defined, otherwise, it will be an object of the single test defined. The object will mirror exactly what was provided in the define method.
Returns: Object|Array
study.assignments()
const study = new Study();study.define({ name: 'new-homepage', buckets: { variantA: { weight: 1 }, }});// Capture assignmentsstudy.assign();// Retrieve all of the bucket assignments for the userconst buckets = study.assignments();assert.strictEqual(buckets['new-homepage'], 'variantA');
This provides the user with all of the bucket assignments for the current user.
The returned information will be an object whose keys will be test names and values will be the current bucket assigned to the user.
// Example return{ 'new-homepage': 'variantA', 'some-test': 'some-bucket',}
Returns: Object|Array
study.extendAssignments
Extending assignments can be a useful way to augment your Study implementation with third party software.
const study = new Study();// Create a function that will modify assignments before you call `assignments`study.extendAssignments = (assignments) => Object.assign(assignments, { foo: 'bar' })// Retrieve all of the bucket assignments for the userconst buckets = study.assignments();assert.strictEqual(buckets['foo'], 'bar');
A more practical example could be to implement with a third party AB testing platform like Optimizely (This uses pseudo code for brevity)
study.extendAssignments = (assignments) => { if (window.optimizely) for (const experiment in optimizely.experiments()) assignments[experiment.name] = experiment.bucket return assignments}
Returns: Object
Guide/FAQ
CSS Driven Tests
Tests logic may be potentially powered on solely CSS. Upon calling assign, if the script is running in the browser, a class per test will be added to the body tag with the test name and bucket in BEM syntax.
<body class="new-homepage--variantA"> <!-- Could be new-homepage--variantB -->
.new-homepage--variantA { /* Write custom styles for the new homepage test */}
Storing metadata associated with tests
Each bucket provided may have additional metadata associated with it, and may have its value retrieved by retrieving the assignments and definitions.
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.