The Bayesian failure probability inference (BFPI) framework provides a well-established Bayesian approach to quantifying our epistemic uncertainty about the failure probability resulting from a limited number of performance function evaluations. However, it is still challenging to perform Bayesian active learning of the failure probability by taking advantage of the BFPI framework. In this work, three Bayesian active learning methods are proposed under the name ‘partially Bayesian active learning cubature’ (PBALC), based on a cleaver use of the BFPI framework for structural reliability analysis, especially when small failure probabilities are involved. Since the posterior variance of the failure probability is computationally expensive to evaluate, the underlying idea is to exploit only the posterior mean of the failure probability to design two critical components for Bayesian active learning, i.e., the stopping criterion and the learning function. On this basis, three sets of stopping criteria and learning functions are proposed, resulting in the three proposed methods PBALC1, PBALC2 and PBALC3. Furthermore, the analytically intractable integrals involved in the stopping criteria are properly addressed from a numerical point of view. Five numerical examples are studied to demonstrate the performance of the three proposed methods. It is found empirically that the proposed methods can assess very small failure probabilities and significantly outperform several existing methods in terms of accuracy and efficiency.