Usage & API reference

Usage

To use the different functionalities/modules of library, create an object of class ABTest with passing class parameters,

  • df = A dataframe which has Users, Response column and Group column.

  • response_column = A binary valued column’s name which has 1 or 0 correspond for the users which being converted or not respectively.

  • group_column = A binary valued column’s name which will indicate the user belong to which group A or B group.

  • labels = (Optional) An array with two string values which will label of the group_column.

Example Code

Below is example code snippet of the functionality and sample dataframe.

from ab_testing import ABTest
from ab_testing.data import Dataset

df = Dataset().data()

abtest_obj = ABTest(df,response_column='Response',group_column='Group')

print(abtest_obj.conversion_rate(),'\n','-'*10)
print(abtest_obj.significance_test(),'\n','-'*10)
print(df.head())

Output:

  Conversion Rate Standard Deviation Standard Error
A          20.20%              0.401          0.018
B          22.20%              0.416         0.0186
 ----------
z statistic: -0.77      p-value: 0.439
Confidence Interval 95% for A group: 16.68% to 23.72%
Confidence Interval 95% for B group: 18.56% to 25.84%

The Group A fail to perform significantly different than group B.
The P-Value of the test is 0.439 which is above 0.05, hence Null hypothesis Hₒ cannot be rejected.
 ----------
        Users  Response Group
0  IS36FC7AQJ         0     A
1  LZW2YNYHZG         1     A
2  9588IGN0RN         1     A
3  HSAH1TYQFF         1     A
4  5D9G147941         0     A

APIs/Functions

Conversion rate conversion_rate

It provides the conversion rate along with standard deviation and standard error values for each group. It returns the these values in table format in pandas dataframe object.

Note

abtest_obj.conversion_rate()

Output:

Conversion Rate Standard Deviation Standard Error
A          20.20%              0.401          0.018
B          22.20%              0.416         0.0186

Significance test significance_test

This provides the significance test report along with conclusive statement. It provides following information as String,

  • Confidence Interval of 95% for each group.

  • P-Value for between group A and B.

  • Z-statistic for between group A and B.

significance_test parameter,

  • threshold = (Optional) To set the P-Value threshold for the significance test, a float value between 0 to 1.

Note

abtest_obj.significance_test()

Output:

z statistic: -0.77      p-value: 0.439
Confidence Interval 95% for A group: 16.68% to 23.72%
Confidence Interval 95% for B group: 18.56% to 25.84%

The Group A fail to perform significantly different than group B.
The P-Value of the test is 0.439 which is above 0.05, hence Null hypothesis Hₒ
cannot be rejected.