Relative unconstrained Least-Squares Importance Fitting (RuLSIF)
Introduction
RuLSIF is an algorithm to directly estimate the relative denisty-ratio:
\(r_{\alpha}({\mathbf x}) = \frac{p({\mathbf x})}{\alpha p({\mathbf x}) + (1 - \alpha)q({\mathbf x})}\)
where \(0 \leq \alpha < 1\) is a parameter.
In addition, using RuLSIF, the relative Pearson divergence (rPE)
\(\mathrm{PE}_{\alpha}[p({\mathbf x}) || q({\mathbf x})] = \frac{1}{2}\int \left(\frac{p({\mathbf x})}{\alpha p({\mathbf x}) + (1 - \alpha)q({\mathbf x})} - 1\right)^2 (\alpha p({\mathbf x}) + (1 - \alpha)q({\mathbf x})) \mathrm{d}{\mathbf x}\)
can be efficiently estimated.
The Matlab code provides the function that computes the relative density-ratio and relative Pearson divergence (rPE).
Download
Examples (Toy data)
Same distribution
 |
Different distribution
 |
Acknowledgement
I am grateful to Prof. Masashi Sugiyama for his support in developing this software.
Contact
I am happy to have any kind of feedbacks. E-mail: yamada AT sg DOT cs DOT titech DOT ac DOT jp
Reference
Yamada, M., Suzuki, T., Kanamori, T., Hachiya, H., & Sugiyama, M.
Relative density-ratio estimation for robust distribution comparison.
In X. XXX, Y. YYY, and Z. ZZZ (Eds.), Advances in Neural Information Processing Systems 24, pp.xxx-xxx, 2011.
(Presented at Neural Information Processing Systems (NIPS2011), Granada, Spain, Dec. 13-15, 2011)