Relative unconstrained Least-Squares Importance Fitting (RuLSIF)


RuLSIF is an algorithm to directly estimate the relative denisty-ratio:

\(r_{\alpha}({\mathbf x}) = \frac{p({\mathbf x})}{\alpha p({\mathbf x}) + (1 - \alpha)q({\mathbf x})}\)

where \(0 \leq \alpha < 1\) is a parameter.

In addition, using RuLSIF, the relative Pearson divergence (rPE)

\(\mathrm{PE}_{\alpha}[p({\mathbf x}) || q({\mathbf x})] = \frac{1}{2}\int \left(\frac{p({\mathbf x})}{\alpha p({\mathbf x}) + (1 - \alpha)q({\mathbf x})} - 1\right)^2 (\alpha p({\mathbf x}) + (1 - \alpha)q({\mathbf x})) \mathrm{d}{\mathbf x}\)

can be efficiently estimated.

The Matlab code provides the function that computes the relative density-ratio and relative Pearson divergence (rPE).


Examples (Toy data)

Same distribution

Same distribution 

Different distribution

Different distribution 


I am grateful to Prof. Masashi Sugiyama for his support in developing this software.


I am happy to have any kind of feedbacks. E-mail: yamada AT sg DOT cs DOT titech DOT ac DOT jp


Yamada, M., Suzuki, T., Kanamori, T., Hachiya, H., & Sugiyama, M.
Relative density-ratio estimation for robust distribution comparison.
In X. XXX, Y. YYY, and Z. ZZZ (Eds.), Advances in Neural Information Processing Systems 24,, 2011.
(Presented at Neural Information Processing Systems (NIPS2011), Granada, Spain, Dec. 13-15, 2011)