Repository navigation
[RF] RooMultiPdf does not respect extended PDF yields for category sampling #22916
Description
When generating toy data from RooMultiPdf with extended PDFs, the category proportions are not weighted by the yields of the component PDFs, whereas RooSimultaneous.generate() does produce category splits proportional to yields.
The reason seems to be that RooSimultaneous implements a specialized generator context (RooSimGenContext) that computes the relative category probabilities from the extended PDF yields and samples the category accordingly. RooMultiPdf does not override genContext()/autoGenContext(), so it falls back to the base RooAbsPdf::generate() behavior and the index category is treated as a fixed parameter rather than a sampled observable.
My question is whether this is intended or not. Either RooMultiPdf is deliberately not meant for category-aware generation, or the missing generator context is an oversight that could be addressed. I'd appreciate clarification on which of the two it is.
The example below reproduces the difference: with yields 100 and 200, RooSimultaneous gives a ~1:2 split while RooMultiPdf does not.
Reproducer
import ROOT
x = ROOT.RooRealVar("x","x",-10,10)
y = ROOT.RooRealVar("y","y",-10,10)
gauss = ROOT.RooGaussian("gauss","gauss", x, 0,1)
gauss1 = ROOT.RooGaussian("gauss1","gauss1", y, 0,1)
lam = ROOT.RooRealVar("lam","lam",-1.,-10.,0.)
exp = ROOT.RooExponential("exp","exp",y,lam)
cat = ROOT.RooCategory("cat","cat")
cat.defineType("gauss",0)
cat.defineType("gauss1",1)
nGauss = ROOT.RooRealVar("nGauss","expected events (gauss)",100,0,1e6)
nGauss1 = ROOT.RooRealVar("nGauss1","expected events (gauss)",200,0,1e6)
gaussExt = ROOT.RooExtendPdf("gaussExt","extended gauss",gauss,nGauss)
gaussExt1 = ROOT.RooExtendPdf("gaussExt1","extended gauss",gauss1,nGauss1)
simPdf = ROOT.RooSimultaneous("simPdf","simultaneous pdf",cat)
simPdf.addPdf(gaussExt,"gauss")
simPdf.addPdf(gaussExt1,"gauss1")
obs_sim = ROOT.RooArgSet(x,y,cat)
simData = simPdf.generate(obs_sim, 1_000)
pdfCat = ROOT.RooCategory("pdfCat","pdfCat")
pdfCat.defineType("gauss",0)
pdfCat.defineType("gauss1",1)
pdfList = ROOT.RooArgList(gaussExt, gaussExt1)
multiPdf = ROOT.RooMultiPdf("multiPdf","multi pdf",pdfCat,pdfList)
obs_multi = ROOT.RooArgSet(x,y,pdfCat)
multiData = multiPdf.generate(obs_multi, 1_000)
simSubData = simData.split(cat)
multiSubData = multiData.split(pdfCat)
print("x-mean, simData:",simSubData[0].to_pandas()["x"].mean())
print("x-mean, multiData:",multiSubData[0].to_pandas()["x"].mean())
print("y-mean, simData:",simSubData[1].to_pandas()["y"].mean())
print("y-mean, multiData:",multiSubData[1].to_pandas()["y"].mean())
n_simData = [ds.numEntries() for ds in simSubData]
n_simData = [n/sum(n_simData) for n in n_simData]
n_multiData = [ds.numEntries() for ds in multiSubData]
n_multiData = [n/sum(n_multiData) for n in n_multiData]
print(f"SimData split: {100*n_simData[0]:.2f}|{100*n_simData[1]:.2f} ")
print(f"MultiData split: {100*n_multiData[0]:.2f}|{100*n_multiData[1]:.2f} ")Version
ROOT v6.41.01
Built for linuxarm64 on Jun 22 2026, 09:59:52
From heads/master@6bf5355c
With c++ (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0 std201703
Binary directory: /usr/share/root/bin
build from source in Ubuntu 24.04 docker container (arm64) on macOS.
Activity
guitargeek commentedon Aug 3, 2026
Hi, thanks for the question and the detailed issue!
I looked into this, and I don't think this is a bug. I think
RooMultiPdfandRooSimultaneousare solving genuinely different statistical problems, and the yield-weighted category sampling that's correct forRooSimultaneousis not meaningful forRooMultiPdf.RooSimultaneous: the index category is an observable. Each event in the data genuinely carries a channel label, and with extended components the model is the joint density that we can meaningfully sample from.RooMultiPdf: the index is a parameter, not an observable. It selects one ofKcompeting modelsf_1, …, f_Kfor the same data. This is the discrete profiling / envelope method first suggested by CMS (theRooMultiPdfclass was upstreamed from the CMS Combine framework), where the index is profiled over like any other nuisance parameter, with a penalty to account for the extra freedom.So given that for the
RooMultiPdfcase, it's a parameter that we have no prior knowledge for, the uniform sampling that you observed is maybe the best thatgenerate()can do. However, I don't see any real analysis workflow that would require marginalizing over the category in a singlegenerate()call. So I think for now the best thing to do is to forbid it to avoid any confusion. I opened a PR for this:What do you think?
cburgard commentedon Aug 3, 2026
I have a somethat more fundamental question about this.
Is it really the case that the only real difference between RooMultiPdf and RooSimultaneous is that one of them treats the categorical variable as an observable (RooSimultaneous) and the other treats the categorical variable as a parameter (RooMultiPdf)?
Because to me, that seems to violate one of the fundamental design decisions of RooFit, namely that the role of any one variable (parameter or observable) is only uniquely identified once the likelihood is built (either the category shows up in the data and is an observable, or it does not and is a parameter).
If that is the only difference - why did RooMultiPdf ever get added rather than extending RooSimultaneous to cover also the RooMultiPdf use case? Is this a historical oversight, or is there a good reason to do it this way? And do you think this is something that could still be facilitated by e.g. extending RooSimultaneous to also cover the functionality of RooMultiPdf and upgrading the "older" models with RooMultiPdf to RooSimultaneous using schema evolution, or are we stuck with this situation now for the time being?
[RF] Add generator context for RooMultiPdf category-aware generation
[RF] Compile parameter-index RooSimultaneous as a switch pdf
[RF] Support parameter-index RooSimultaneous as top-level fit pdf
[RF] Compile parameter-index RooSimultaneous as a switch pdf
[RF] Support parameter-index RooSimultaneous as top-level fit pdf