Compact Muon Solenoid
LHC, CERN
Visit us: CMS Public Website, CMS Physics ;
Contact us: CMS Publications Committee
| CMS-MLG-23-005 ; CERN-EP-2025-005 | ||
| Development of systematic uncertainty-aware neural network trainings for binned-likelihood analyses at the LHC | ||
| CMS Collaboration | ||
| 18 February 2025 | ||
| Eur. Phys. J. C 85 (2025) 1360 | ||
| Abstract: We propose a neural network training method capable of accounting for the effects of systematic variations of the data model in the training process and describe its extension towards neural network multiclass classification. The procedure is evaluated on the realistic case of the measurement of Higgs boson production via gluon fusion and vector boson fusion in the $ \tau\tau $ decay channel at the CMS experiment. The neural network output functions are used to infer the signal strengths for inclusive production of Higgs bosons as well as for their production via gluon fusion and vector boson fusion. We observe improvements of 12 and 16% in the uncertainty in the signal strengths for gluon and vector-boson fusion, respectively, compared with a conventional neural network training based on cross-entropy. | ||
| Links: e-print arXiv:2502.13047 [hep-ex] (PDF) ; CDS record ; inSPIRE record ; CADI line (restricted) ; | ||
| Figures | |
png pdf |
Figure 1:
Flow chart of a $ \text{CENNT} $ (upper) and $ \text{SANNT} $ (lower). In the figure $ D_{i} $ denotes the data set; $ n $ ($ d $) the number of events (observables) in the initial data set $ D_{X} $; $ l $ the number of classes after event classification; and $ h $ the number of histogram bins entering the statistical inference of the POIs. The function symbol $ \mathbb{P} $ represents the multinomial distribution, and the symbol $ \mathcal{L} $ has been defined in Eq. (3). |
png pdf |
Figure 1-a:
Flow chart of a $ \text{CENNT} $ (upper) and $ \text{SANNT} $ (lower). In the figure $ D_{i} $ denotes the data set; $ n $ ($ d $) the number of events (observables) in the initial data set $ D_{X} $; $ l $ the number of classes after event classification; and $ h $ the number of histogram bins entering the statistical inference of the POIs. The function symbol $ \mathbb{P} $ represents the multinomial distribution, and the symbol $ \mathcal{L} $ has been defined in Eq. (3). |
png pdf |
Figure 1-b:
Flow chart of a $ \text{CENNT} $ (upper) and $ \text{SANNT} $ (lower). In the figure $ D_{i} $ denotes the data set; $ n $ ($ d $) the number of events (observables) in the initial data set $ D_{X} $; $ l $ the number of classes after event classification; and $ h $ the number of histogram bins entering the statistical inference of the POIs. The function symbol $ \mathbb{P} $ represents the multinomial distribution, and the symbol $ \mathcal{L} $ has been defined in Eq. (3). |
png pdf |
Figure 2:
Custom functions $ \mathcal{B}_{i} $ for the backward pass of the backpropagation algorithm, as used (left) in Ref. [14] and (right) in this paper. In the first row of each sub-figure, the same 20 random samples of a simple setup of pseudo-experiments, as described in Section 3.2, are shown. In the second row the resulting histogram $ H $, in the third and fourth rows the functions $ \mathcal{B}_{0} $ and $ \mathcal{B}_{1} $ for the individual bins $ H_{0} $ and $ H_{1} $, respectively, and in the last row the collective effect of $ \sum\mathcal{B}_{i} $ are shown. |
png pdf |
Figure 2-a:
Custom functions $ \mathcal{B}_{i} $ for the backward pass of the backpropagation algorithm, as used (left) in Ref. [14] and (right) in this paper. In the first row of each sub-figure, the same 20 random samples of a simple setup of pseudo-experiments, as described in Section 3.2, are shown. In the second row the resulting histogram $ H $, in the third and fourth rows the functions $ \mathcal{B}_{0} $ and $ \mathcal{B}_{1} $ for the individual bins $ H_{0} $ and $ H_{1} $, respectively, and in the last row the collective effect of $ \sum\mathcal{B}_{i} $ are shown. |
png pdf |
Figure 2-b:
Custom functions $ \mathcal{B}_{i} $ for the backward pass of the backpropagation algorithm, as used (left) in Ref. [14] and (right) in this paper. In the first row of each sub-figure, the same 20 random samples of a simple setup of pseudo-experiments, as described in Section 3.2, are shown. In the second row the resulting histogram $ H $, in the third and fourth rows the functions $ \mathcal{B}_{0} $ and $ \mathcal{B}_{1} $ for the individual bins $ H_{0} $ and $ H_{1} $, respectively, and in the last row the collective effect of $ \sum\mathcal{B}_{i} $ are shown. |
png pdf |
Figure 3:
Evolution of the loss functions CE, $ \Delta r_{s}^{\text{stat}} $, and $ \Delta r_{s} $ as used (left) in Ref. [14] and (right) for this paper. In the upper panels the evolution of $ \hat{y}( \cdot ) $ for randomly selected 50 (blue) signal and 50 (orange) background samples during training is shown. Dashed horizontal lines indicate the boundaries of the histogram bins $ H_{i} $. The gray shaded area indicates the pre-training. In the second and third panels from above the evolution of CE and $ \Delta r_{s}^{\text{stat}} $ is shown. In the lowest panels the evolution of $ L_{\text{SANNT}}=\Delta r_{s} $ is shown. The evaluation on the training (validation) data set is indicated in blue (orange). The evaluation of the correspondingly inactive loss function, during or after pre-training, evaluated on the validation data set is indicated by the dashed orange curves. |
png pdf |
Figure 3-a:
Evolution of the loss functions CE, $ \Delta r_{s}^{\text{stat}} $, and $ \Delta r_{s} $ as used (left) in Ref. [14] and (right) for this paper. In the upper panels the evolution of $ \hat{y}( \cdot ) $ for randomly selected 50 (blue) signal and 50 (orange) background samples during training is shown. Dashed horizontal lines indicate the boundaries of the histogram bins $ H_{i} $. The gray shaded area indicates the pre-training. In the second and third panels from above the evolution of CE and $ \Delta r_{s}^{\text{stat}} $ is shown. In the lowest panels the evolution of $ L_{\text{SANNT}}=\Delta r_{s} $ is shown. The evaluation on the training (validation) data set is indicated in blue (orange). The evaluation of the correspondingly inactive loss function, during or after pre-training, evaluated on the validation data set is indicated by the dashed orange curves. |
png pdf |
Figure 3-b:
Evolution of the loss functions CE, $ \Delta r_{s}^{\text{stat}} $, and $ \Delta r_{s} $ as used (left) in Ref. [14] and (right) for this paper. In the upper panels the evolution of $ \hat{y}( \cdot ) $ for randomly selected 50 (blue) signal and 50 (orange) background samples during training is shown. Dashed horizontal lines indicate the boundaries of the histogram bins $ H_{i} $. The gray shaded area indicates the pre-training. In the second and third panels from above the evolution of CE and $ \Delta r_{s}^{\text{stat}} $ is shown. In the lowest panels the evolution of $ L_{\text{SANNT}}=\Delta r_{s} $ is shown. The evaluation on the training (validation) data set is indicated in blue (orange). The evaluation of the correspondingly inactive loss function, during or after pre-training, evaluated on the validation data set is indicated by the dashed orange curves. |
png pdf |
Figure 4:
Expected distributions of $ \hat{y}( \cdot ) $ for a binary classification task separating $ S $ from $ B $, for a (left) $ \text{CENNT} $ and (right) $ \text{SANNT} $, prior to any fit to $ D_{H}^{\mathcal{A}} $. The individual distributions for $ S $ and $ B $ are shown by the nonstacked open blue and filled orange histograms, respectively. In the lower panels of the figures the expected values of $ S/B+ $ 1 are shown. The gray bands correspond to the combined statistical and systematic uncertainty in $ B $. The boundaries of the histogram bins $ H_{i} $ are also given on an extra horizontal axis. |
png pdf |
Figure 4-a:
Expected distributions of $ \hat{y}( \cdot ) $ for a binary classification task separating $ S $ from $ B $, for a (left) $ \text{CENNT} $ and (right) $ \text{SANNT} $, prior to any fit to $ D_{H}^{\mathcal{A}} $. The individual distributions for $ S $ and $ B $ are shown by the nonstacked open blue and filled orange histograms, respectively. In the lower panels of the figures the expected values of $ S/B+ $ 1 are shown. The gray bands correspond to the combined statistical and systematic uncertainty in $ B $. The boundaries of the histogram bins $ H_{i} $ are also given on an extra horizontal axis. |
png pdf |
Figure 4-b:
Expected distributions of $ \hat{y}( \cdot ) $ for a binary classification task separating $ S $ from $ B $, for a (left) $ \text{CENNT} $ and (right) $ \text{SANNT} $, prior to any fit to $ D_{H}^{\mathcal{A}} $. The individual distributions for $ S $ and $ B $ are shown by the nonstacked open blue and filled orange histograms, respectively. In the lower panels of the figures the expected values of $ S/B+ $ 1 are shown. The gray bands correspond to the combined statistical and systematic uncertainty in $ B $. The boundaries of the histogram bins $ H_{i} $ are also given on an extra horizontal axis. |
png pdf |
Figure 5:
Impacts for the 20 nuisance parameters $ \theta_{j} $ with the largest impacts on $ r_{s} $. The gray lines refer to the $ \text{CENNT} $ and the colored bars to the $ \text{SANNT} $. The impacts can be read from the $ x $ axis. Labels shown on the $ y $ axis for each $ \theta_{j} $ are defined in Table 1. The entries are ordered by decreasing magnitude for $ \text{CENNT} $ when moving from the top to the bottom of the figure. The panel on the right shows the relative change of the symmetrized impact when moving from $ \text{CENNT} $ to $ \text{SANNT} $. A more detailed discussion is given in the text. |
png pdf |
Figure 6:
Negative log of the profile likelihood $ -2\Delta\log\mathcal{L} $ as a function of $ r_{s} $, taking into account (red) all and (blue) only the statistical uncertainties in $ \Delta r_{s} $. The results obtained from the $ \text{CENNT} $ are indicated by the dashed lines, and the median expected results of ensembles of 100 repetitions of the $ \text{SANNT} $ varying random initializations are indicated by the continuous lines. The red and blue shaded bands surrounding the median expectations indicate 68% confidence intervals (CI) from these ensembles. The lower panels show the distributions underlying these CI. |
png pdf |
Figure 7:
Expected distributions of $ \hat{y}_{l}( \cdot ) $ for multiclass classification, based on seven event classes, as used for a differential STXS cross section measurement of H production in Ref. [8], prior to any fit to $ D_{H}^{\mathcal{A}} $. In the upper (lower) part of the figure the results obtained after $ \text{CENNT} $ ($ \text{SANNT} $) are shown. The background processes of $ \Omega_{X} $ are indicated by stacked, colored, filled histograms. The expected $ \mathrm{g}\mathrm{g}\mathrm{H} $ and $ \mathrm{q}\mathrm{q}\mathrm{H} $ contributions are indicated by the nonstacked open histograms. In the lower panels of the figure the expected values of $ S/B+ $ 1 are shown. The gray bands correspond to the combined statistical and systematic uncertainty in the background model. |
png pdf |
Figure 7-a:
Expected distributions of $ \hat{y}_{l}( \cdot ) $ for multiclass classification, based on seven event classes, as used for a differential STXS cross section measurement of H production in Ref. [8], prior to any fit to $ D_{H}^{\mathcal{A}} $. In the upper (lower) part of the figure the results obtained after $ \text{CENNT} $ ($ \text{SANNT} $) are shown. The background processes of $ \Omega_{X} $ are indicated by stacked, colored, filled histograms. The expected $ \mathrm{g}\mathrm{g}\mathrm{H} $ and $ \mathrm{q}\mathrm{q}\mathrm{H} $ contributions are indicated by the nonstacked open histograms. In the lower panels of the figure the expected values of $ S/B+ $ 1 are shown. The gray bands correspond to the combined statistical and systematic uncertainty in the background model. |
png pdf |
Figure 7-b:
Expected distributions of $ \hat{y}_{l}( \cdot ) $ for multiclass classification, based on seven event classes, as used for a differential STXS cross section measurement of H production in Ref. [8], prior to any fit to $ D_{H}^{\mathcal{A}} $. In the upper (lower) part of the figure the results obtained after $ \text{CENNT} $ ($ \text{SANNT} $) are shown. The background processes of $ \Omega_{X} $ are indicated by stacked, colored, filled histograms. The expected $ \mathrm{g}\mathrm{g}\mathrm{H} $ and $ \mathrm{q}\mathrm{q}\mathrm{H} $ contributions are indicated by the nonstacked open histograms. In the lower panels of the figure the expected values of $ S/B+ $ 1 are shown. The gray bands correspond to the combined statistical and systematic uncertainty in the background model. |
png pdf |
Figure 8:
Negative log of the profile likelihood $ -2\Delta\log\mathcal{L} $ as a function of $ r_{s} $, for a differential STXS cross section measurement of H production in the $ \mathrm{H}\to\tau\tau $ decay channel, taking (red) all and (blue) only the statistical uncertainties in $ \Delta r_{s} $ into account. In the left plot $ r_{\text{inc}} $ for an inclusive measurement is shown, and in the middle and right plots $ r_{\mathrm{g}\mathrm{g}\mathrm{H}} $ and $ r_{\mathrm{q}\mathrm{q}\mathrm{H}} $ for a combined differential STXS measurement of these two contributions to the signal in two bins are shown. The results as obtained from the $ \text{CENNT} $ are indicated by the dashed lines, and the median expected results of ensembles of 100 repetitions of the $ \text{SANNT} $ varying random initializations are indicated by the continuous lines. The red and blue shaded bands surrounding the median expectations indicate 68% confidence intervals (CI) from these ensembles. |
png pdf |
Figure 8-a:
Negative log of the profile likelihood $ -2\Delta\log\mathcal{L} $ as a function of $ r_{s} $, for a differential STXS cross section measurement of H production in the $ \mathrm{H}\to\tau\tau $ decay channel, taking (red) all and (blue) only the statistical uncertainties in $ \Delta r_{s} $ into account. In the left plot $ r_{\text{inc}} $ for an inclusive measurement is shown, and in the middle and right plots $ r_{\mathrm{g}\mathrm{g}\mathrm{H}} $ and $ r_{\mathrm{q}\mathrm{q}\mathrm{H}} $ for a combined differential STXS measurement of these two contributions to the signal in two bins are shown. The results as obtained from the $ \text{CENNT} $ are indicated by the dashed lines, and the median expected results of ensembles of 100 repetitions of the $ \text{SANNT} $ varying random initializations are indicated by the continuous lines. The red and blue shaded bands surrounding the median expectations indicate 68% confidence intervals (CI) from these ensembles. |
png pdf |
Figure 8-b:
Negative log of the profile likelihood $ -2\Delta\log\mathcal{L} $ as a function of $ r_{s} $, for a differential STXS cross section measurement of H production in the $ \mathrm{H}\to\tau\tau $ decay channel, taking (red) all and (blue) only the statistical uncertainties in $ \Delta r_{s} $ into account. In the left plot $ r_{\text{inc}} $ for an inclusive measurement is shown, and in the middle and right plots $ r_{\mathrm{g}\mathrm{g}\mathrm{H}} $ and $ r_{\mathrm{q}\mathrm{q}\mathrm{H}} $ for a combined differential STXS measurement of these two contributions to the signal in two bins are shown. The results as obtained from the $ \text{CENNT} $ are indicated by the dashed lines, and the median expected results of ensembles of 100 repetitions of the $ \text{SANNT} $ varying random initializations are indicated by the continuous lines. The red and blue shaded bands surrounding the median expectations indicate 68% confidence intervals (CI) from these ensembles. |
png pdf |
Figure 8-c:
Negative log of the profile likelihood $ -2\Delta\log\mathcal{L} $ as a function of $ r_{s} $, for a differential STXS cross section measurement of H production in the $ \mathrm{H}\to\tau\tau $ decay channel, taking (red) all and (blue) only the statistical uncertainties in $ \Delta r_{s} $ into account. In the left plot $ r_{\text{inc}} $ for an inclusive measurement is shown, and in the middle and right plots $ r_{\mathrm{g}\mathrm{g}\mathrm{H}} $ and $ r_{\mathrm{q}\mathrm{q}\mathrm{H}} $ for a combined differential STXS measurement of these two contributions to the signal in two bins are shown. The results as obtained from the $ \text{CENNT} $ are indicated by the dashed lines, and the median expected results of ensembles of 100 repetitions of the $ \text{SANNT} $ varying random initializations are indicated by the continuous lines. The red and blue shaded bands surrounding the median expectations indicate 68% confidence intervals (CI) from these ensembles. |
png pdf |
Figure A1:
Evolution of the loss functions CE, $ \Delta r_{s}^{\text{stat}} $, and $ \Delta r_{s} $, as used for this paper. Instead of the custom functions $ \mathcal{B}_{i} $ the identity operation (the so-called straight-through estimator) is used for $ \text{SANNT} $. In the upper panel, the evolution of $ \hat{y} $ for randomly selected 50 (blue) signal and 50 (orange) background samples during training is shown. The gray shaded area indicates the pre-training. In the second panel from above the evolution of CE is shown. Though not actively used for the $ \text{SANNT} \Delta r_{s}^{\text{stat}} $ is also shown, in the third panel from above. In the lower panel, the evolution of $ \Delta r_{s} $ is shown. The evaluation on the training (validation) data set is indicated in blue (orange). The evolution of inactive loss functions, evaluated on the validation data set, is indicated by the dashed orange curves. |
| Tables | |
png pdf |
Table 1:
Association of nuisance parameters $ \theta_{j} $ with the systematic variations they refer to for the 20 parameters with the largest impacts on $ r_{s} $, as shown in Fig. 5. The label of each corresponding uncertainty is given in the first column, the type of uncertainty, process that it applies to, and rank in Fig. 5 are given in the second, third, and fourth columns, respectively. The ``*'' in $ \epsilon_{\tau}^{\mathrm{ID}}( \text{1-prong}^{*}) $ refers to the fact that this is the decay channel with neutral pions in addition to the charged prong. The symbol EMB refers to $ \tau $-embedded event samples. A more detailed discussion is given in the text. |
png pdf |
Table 2:
Expected combined statistical and systematic uncertainties $ \Delta r_{s} $ and statistical uncertainties $ \Delta r_{s}^{\text{stat}} $ in the parameters $ r_{\text{inc}} $ for an inclusive, and $ r_{\mathrm{g}\mathrm{g}\mathrm{H}} $ and $ r_{\mathrm{q}\mathrm{q}\mathrm{H}} $ for a differential STXS, cross section measurement of H production in the $ \mathrm{H}\to\tau\tau $ decay channel, as obtained from fits to $ D_{H}^{\mathcal{A}} $. In the second (third) column the results after $ \text{SANNT} $ ($ \text{CENNT} $) are shown. |
| Summary |
| We have proposed a neural network training method capable of accounting for the effects of systematic variations of the data model used in the training process and described its extension to neural network multiclass classification. The procedure has been evaluated on the realistic case of the measurement of Higgs boson production via gluon fusion and vector boson fusion in the $ \tau\tau $ decay channel at the CMS experiment. The neural network output functions are used to infer the signal strengths for inclusive production of Higgs bosons as well as for their production via gluon fusion and vector boson fusion. We observe improvements of 12 and 16% in the uncertainty in the signal strengths for gluon and vector-boson fusion, respectively, compared with a conventional neural network training based on cross-entropy. This is the first time that a neural network training capable of accounting for the effects of systematic variations has been demonstrated on a data model of this complexity and the first time that such a training has been applied to multiclass classification. |
| References | ||||
| 1 | ATLAS Collaboration | Observation of a new particle in the search for the standard model Higgs boson with the ATLAS detector at the LHC | PLB 716 (2012) 1 | 1207.7214 |
| 2 | CMS Collaboration | Observation of a new boson at a mass of 125 GeV with the CMS experiment at the LHC | PLB 716 (2012) 30 | CMS-HIG-12-028 1207.7235 |
| 3 | CMS Collaboration | Observation of a new boson with mass near 125 GeV in pp collisions at $ \sqrt{s} = $ 7 and 8 TeV | JHEP 06 (2013) 081 | CMS-HIG-12-036 1303.4571 |
| 4 | CMS Collaboration | A portrait of the higgs boson by the CMS experiment ten years after the discovery | Nature 607 (2022) 60 | CMS-HIG-22-001 2207.00043 |
| 5 | ATLAS Collaboration | Characterising the Higgs boson with ATLAS data from Run 2 of the LHC | Phys. Rept. 11 (2024) 001 | 2404.05498 |
| 6 | ATLAS Collaboration | Measurements of Higgs bosons decaying to bottom quarks from vector boson fusion production with the ATLAS experiment at $ \sqrt{s}= $ 13 TeV | EPJC 81 (2021) 537 | 2011.08280 |
| 7 | CMS Collaboration | Measurement of simplified template cross sections of the Higgs boson produced in association with W or Z bosons in the $ \mathrm{H}\to\mathrm{b}\mathrm{b} $ decay channel in proton-proton collisions at $ \sqrt{s}= $ 13 TeV | PRD 109 (2024) 092011 | CMS-HIG-20-001 2312.07562 |
| 8 | CMS Collaboration | Measurements of Higgs boson production in the decay channel with a pair of $ \tau $ leptons in proton-proton collisions at $ \sqrt{s}= $ 13 TeV | EPJC 83 (2023) 562 | CMS-HIG-19-010 2204.12957 |
| 9 | ATLAS Collaboration | Differential cross-section measurements of Higgs boson production in the $ \mathrm{H}\to\tau^+\tau^- $ decay channel in pp collisions at $ \sqrt{s}= $ 13 TeV with the ATLAS detector | Submitted to JHEP, 2024 | 2407.16320 |
| 10 | I. J. Good | Rational decisions | J. Royal Stat. Soc. B 14 (1952) 107 | |
| 11 | R. M. Neal | Computing likelihood functions for high-energy physics experiments when distributions are defined by simulators with nuisance parameters | in PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics, 2008 link |
|
| 12 | K. Cranmer, J. Pavez, and G. Louppe | Approximating likelihood ratios with calibrated discriminative classifiers | 1506.02169 | |
| 13 | P. De Castro and T. Dorigo | INFERNO: Inference-aware neural optimisation | Comput. Phys. Commun. 244 (2019) 170 | 1806.04743 |
| 14 | S. Wunsch, S. Jörger, R. Wolf, and G. Quast | Optimal statistical inference in the presence of systematic uncertainties using neural network optimization based on binned Poisson likelihoods with nuisance parameters | Comput. Softw. Big Sci. 5 (2021) 4 | 2003.07186 |
| 15 | N. Simpson and L. Heinrich | neos: End-to-end-optimised summary statistics for high energy physics | J. Phys. Conf. Ser. 2438 (2023) 012105 | 2203.05570 |
| 16 | G. Louppe, M. Kagan, and K. Cranmer | Learning to pivot with adversarial networks | in Advances in Neural Information Processing Systems, 2017 | 1611.01046 |
| 17 | S. Wunsch, S. Jorger, R. Wolf, and G. Quast | Reducing the dependence of the neural network function to systematic uncertainties in the input space | Comput. Softw. Big Sci. 4 (2020) 5 | 1907.11674 |
| 18 | R. A. Fisher | On the mathematical foundations of theoretical statistics | J. R. Stat. Soc. 82 (1922) 87 | |
| 19 | H. Cramér | Mathematical methods of statistics | Princeton University Press, 1946 | |
| 20 | C. R. Rao | Information and the accuracy attainable in the estimation of statistical parameters | in Breakthroughs in statistics, Springer, 1992 | |
| 21 | CMS Collaboration | Performance of the CMS Level-1 trigger in proton-proton collisions at $ \sqrt{s} = $ 13 TeV | JINST 15 (2020) P10017 | CMS-TRG-17-001 2006.10165 |
| 22 | CMS Collaboration | The CMS trigger system | JINST 12 (2017) P01020 | CMS-TRG-12-001 1609.02366 |
| 23 | CMS Collaboration | Performance of the CMS high-level trigger during LHC Run 2 | JINST 19 (2024) P11021 | CMS-TRG-19-001 2410.17038 |
| 24 | CMS Collaboration | The CMS experiment at the CERN LHC | JINST 3 (2008) S08004 | |
| 25 | CMS Collaboration | Particle-flow reconstruction and global event description with the CMS detector | JINST 12 (2017) P10003 | CMS-PRF-14-001 1706.04965 |
| 26 | CMS Collaboration | Technical proposal for the Phase-II upgrade of the Compact Muon Solenoid | CMS Technical Proposal CERN-LHCC-2015-010, CMS-TDR-15-02, 2015 CDS |
|
| 27 | CMS Collaboration | Performance of electron reconstruction and selection with the CMS detector in proton-proton collisions at $ \sqrt{s} = $ 8 TeV | JINST 10 (2015) P06005 | CMS-EGM-13-001 1502.02701 |
| 28 | CMS Collaboration | Electron and photon reconstruction and identification with the CMS experiment at the CERN LHC | JINST 16 (2021) P05014 | CMS-EGM-17-001 2012.06888 |
| 29 | CMS Collaboration | Performance of CMS muon reconstruction in pp collision events at $ \sqrt{s}= $ 7 TeV | JINST 7 (2012) P10002 | CMS-MUO-10-004 1206.4071 |
| 30 | CMS Collaboration | Performance of the CMS muon detector and muon reconstruction with proton-proton collisions at $ \sqrt{s}= $ 13 TeV | JINST 13 (2018) P06015 | CMS-MUO-16-001 1804.04528 |
| 31 | M. Cacciari, G. P. Salam, and G. Soyez | The anti-$ k_{\mathrm{T}} $ jet clustering algorithm | JHEP 04 (2008) 063 | 0802.1189 |
| 32 | M. Cacciari, G. P. Salam, and G. Soyez | FastJet user manual | EPJC 72 (2012) 1896 | 1111.6097 |
| 33 | CMS Collaboration | Identification of heavy-flavour jets with the CMS detector in pp collisions at 13 TeV | JINST 13 (2018) P05011 | CMS-BTV-16-002 1712.07158 |
| 34 | E. Bols et al. | Jet flavour classification using DeepJet | JINST 15 (2020) P12012 | 2008.10519 |
| 35 | CMS Collaboration | Performance of reconstruction and identification of $ \tau $ leptons decaying to hadrons and $ \nu_\tau $ in pp collisions at $ \sqrt{s}= $ 13 TeV | JINST 13 (2018) P10005 | CMS-TAU-16-003 1809.02816 |
| 36 | CMS Collaboration | Identification of hadronic tau lepton decays using a deep neural network | JINST 17 (2022) P07023 | CMS-TAU-20-001 2201.08458 |
| 37 | CMS Collaboration | Performance of the CMS missing transverse momentum reconstruction in pp data at $ \sqrt{s} = $ 8 TeV | JINST 10 (2015) P02006 | CMS-JME-13-003 1411.0511 |
| 38 | D. Bertolini, P. Harris, M. Low, and N. Tran | Pileup per particle identification | JHEP 10 (2014) 059 | 1407.6013 |
| 39 | S. D. Drell and T.-M. Yan | Massive lepton-pair production in hadron-hadron collisions at high energies | PRL 25 (1970) 316 | |
| 40 | CMS Collaboration | An embedding technique to determine $ \tau\tau $ backgrounds in proton-proton collision data | JINST 14 (2019) P06032 | CMS-TAU-18-001 1903.01216 |
| 41 | CMS Collaboration | Measurement of the $ \mathrm{Z}\gamma^{*}\to\tau\tau $ cross section in pp collisions at $ \sqrt{s}= $ 13 TeV and validation of $ \tau $ lepton analysis techniques | EPJC 78 (2018) 708 | CMS-HIG-15-007 1801.03535 |
| 42 | CMS Collaboration | Search for additional neutral MSSM Higgs bosons in the $ \tau\tau $ final state in proton-proton collisions at $ \sqrt{s}= $ 13 TeV | JHEP 09 (2018) 007 | CMS-HIG-17-020 1803.06553 |
| 43 | P. Nason | A new method for combining NLO QCD with shower Monte Carlo algorithms | JHEP 11 (2004) 040 | hep-ph/0409146 |
| 44 | S. Frixione, P. Nason, and C. Oleari | Matching NLO QCD computations with parton shower simulations: the POWHEG method | JHEP 11 (2007) 070 | 0709.2092 |
| 45 | S. Alioli, P. Nason, C. Oleari, and E. Re | NLO Higgs boson production via gluon fusion matched with shower in POWHEG | JHEP 04 (2009) 002 | 0812.0578 |
| 46 | S. Alioli, P. Nason, C. Oleari, and E. Re | A general framework for implementing NLO calculations in shower Monte Carlo programs: the POWHEG BOX | JHEP 06 (2010) 043 | 1002.2581 |
| 47 | S. Alioli et al. | Jet pair production in POWHEG | JHEP 04 (2011) 081 | 1012.3380 |
| 48 | E. Bagnaschi, G. Degrassi, P. Slavich, and A. Vicini | Higgs production via gluon fusion in the POWHEG approach in the SM and in the MSSM | JHEP 02 (2012) 088 | 1111.2854 |
| 49 | J. Alwall et al. | MadGraph 5: Going beyond | JHEP 06 (2011) 128 | 1106.0522 |
| 50 | J. Alwall et al. | The automated computation of tree-level and next-to-leading order differential cross sections, and their matching to parton shower simulations | JHEP 07 (2014) 079 | 1405.0301 |
| 51 | T. Sjöstrand et al. | An introduction to PYTHIA 8.2 | Comput. Phys. Commun. 191 (2015) 159 | 1410.3012 |
| 52 | S. Agostinelli et al. | GEANT 4---a simulation toolkit | NIM A 506 (2003) 250 | |
| 53 | K. Fukushima | Cognitron: A self-organizing multilayered neural network | Biol. Cybernetics 20 (1975) 121 | |
| 54 | V. Nair and G. E. Hinton | Rectified linear units improve restricted boltzmann machines | in Proc. 27th Int. Conf. on Machine Learning, ICML'10, Omnipress, Madison, WI, USA, 2010 link |
|
| 55 | L. Bianchini et al. | Reconstruction of the Higgs mass in events with Higgs bosons decaying into a pair of $ \tau $ leptons using matrix element techniques | NIM A 862 (2017) 54 | 1603.05910 |
| 56 | A. V. Gritsan, R. Röntsch, M. Schulze, and M. Xiao | Constraining anomalous Higgs boson couplings to the heavy flavor fermions using matrix element techniques | PRD 94 (2016) 055023 | 1606.03107 |
| 57 | LHC Higgs Cross Section Working Group | Handbook of LHC Higgs cross sections: 4. Deciphering the nature of the Higgs sector | CERN Yellow Reports:, 2017 Mongraphs 2 (2017) |
1610.07922 |
| 58 | N. Berger et al. | Simplified template cross sections - stage 1.1 | LHC Higgs Cross Section Working Group Report LHCHXSWG-2019-003, DESY-19-070, 2019 link |
1906.02754 |
| 59 | G. Cowan, K. Cranmer, E. Gross, and O. Vitells | Asymptotic formulae for likelihood-based tests of new physics | EPJC 71 (2011) 1554 | 1007.1727 |
| 60 | CMS Collaboration | The CMS statistical analysis and combination tool: Combine | Comput. Softw. Big Sci. 8 (2024) 19 | CMS-CAT-23-001 2404.06614 |
| 61 | R. J. Barlow and C. Beeston | Fitting using finite Monte Carlo samples | Comput. Phys. Commun. 77 (1993) 219 | |
| 62 | CMS Collaboration | CMS luminosity measurements for the 2016 data-taking period | Technical Report , CERN, 2017 CMS-PAS-LUM-17-001 |
CMS-PAS-LUM-17-001 |
| 63 | CMS Collaboration | CMS luminosity measurement for the 2017 data-taking period at $ \sqrt{s} = $ 13 TeV | CMS Physics Analysis Summary, 2018 CMS-PAS-LUM-17-004 |
CMS-PAS-LUM-17-004 |
| 64 | CMS Collaboration | CMS luminosity measurement for the 2018 data-taking period at $ \sqrt{s} = $ 13 TeV | CMS Physics Analysis Summary, 2019 CMS-PAS-LUM-18-002 |
CMS-PAS-LUM-18-002 |
| 65 | CMS Collaboration | Precision luminosity measurement in proton-proton collisions at $ \sqrt{s} = $ 13 TeV in 2015 and 2016 at CMS | EPJC 81 (2021) 800 | CMS-LUM-17-003 2104.01927 |
| 66 | Y. Bengio, N. Léonard, and A. C. Courville | Estimating or propagating gradients through stochastic neurons for conditional computation | 1308.3432 | |
| 67 | S. Wunsch, R. Friese, R. Wolf, and G. Quast | Identifying the relevant dependencies of the neural network response on characteristics of the input space | Comput. Softw. Big Sci. 2 (2018) 5 | 1803.08782 |
| 68 | J. C. Platt and A. H. Barr | Constrained differential optimization | in NIPS, American Institue of Physics, 1987 | |
|
|
Compact Muon Solenoid LHC, CERN |
|
|
|
|
|
|