Usage & API reference¶
Usage¶
To use the different functionalities/modules of library, create an object of class ABTest with passing class parameters,
df
= A dataframe which has Users, Response column and Group column.response_column
= A binary valued column’s name which has 1 or 0 correspond for the users which being converted or not respectively.group_column
= A binary valued column’s name which will indicate the user belong to which group A or B group.labels
= (Optional) An array with two string values which will label of thegroup_column
.
Example Code¶
Below is example code snippet of the functionality and sample dataframe.
from ab_testing import ABTest
from ab_testing.data import Dataset
df = Dataset().data()
abtest_obj = ABTest(df,response_column='Response',group_column='Group')
print(abtest_obj.conversion_rate(),'\n','-'*10)
print(abtest_obj.significance_test(),'\n','-'*10)
print(df.head())
Output:
Conversion Rate Standard Deviation Standard Error
A 20.20% 0.401 0.018
B 22.20% 0.416 0.0186
----------
z statistic: -0.77 p-value: 0.439
Confidence Interval 95% for A group: 16.68% to 23.72%
Confidence Interval 95% for B group: 18.56% to 25.84%
The Group A fail to perform significantly different than group B.
The P-Value of the test is 0.439 which is above 0.05, hence Null hypothesis Hₒ cannot be rejected.
----------
Users Response Group
0 IS36FC7AQJ 0 A
1 LZW2YNYHZG 1 A
2 9588IGN0RN 1 A
3 HSAH1TYQFF 1 A
4 5D9G147941 0 A
APIs/Functions¶
Conversion rate conversion_rate
¶
It provides the conversion rate along with standard deviation and standard error values for each group. It returns the these values in table format in pandas dataframe object.
Note
abtest_obj.conversion_rate()
Output:
Conversion Rate Standard Deviation Standard Error
A 20.20% 0.401 0.018
B 22.20% 0.416 0.0186
Significance test significance_test
¶
This provides the significance test report along with conclusive statement. It provides following information as String,
Confidence Interval of 95% for each group.
P-Value for between group A and B.
Z-statistic for between group A and B.
significance_test
parameter,
threshold
= (Optional) To set the P-Value threshold for the significance test, a float value between 0 to 1.
Note
abtest_obj.significance_test()
Output:
z statistic: -0.77 p-value: 0.439
Confidence Interval 95% for A group: 16.68% to 23.72%
Confidence Interval 95% for B group: 18.56% to 25.84%
The Group A fail to perform significantly different than group B.
The P-Value of the test is 0.439 which is above 0.05, hence Null hypothesis Hₒ
cannot be rejected.