From Equations to Predictions: Machine Learning in Particle Theory#

2.1 Categories of ML#

1. Supervised Learning: In supervised learning, the algorithm learns from labeled data. For instance, we may have simulations of certain particle decay channels labeled as “Channel A�?versus “Channel B,�?and we want to predict which channel future events belong to.
2. Unsupervised Learning: Unsupervised learning attempts to uncover hidden structures in unlabeled data. This could be used to cluster events in collider data to identify spontaneous patterns (e.g., unusual jets).
3. Reinforcement Learning: RL focuses on agents making sequential decisions to maximize a reward signal. Conceptually, it can be employed in automated symbolic manipulations or parameter searches in large theory spaces.

2.2 Basic Terminologies#

�?Features: Individual measurable properties, such as collision energy, momentum of particles, or angles of jets.
�?Labels / Targets: The outcomes or categories we want to predict (e.g., type of boson, presence of new particles).
�?Loss Function: Quantification of how well (or poorly) the model is performing. Minimizing this helps the model adapt to the data.
�?Model Capacity: The “size�?or complexity of the model, which can range from simple linear regressors to complex deep neural networks.
�?Regularization: Techniques to prevent overfitting by penalizing large parameter values or adding constraints.

3.1 Simulation Tools#

Particle theory often depends on Monte Carlo (MC) event generators such as PYTHIA, HERWIG, or SHERPA, which simulate high-energy collisions under various theoretical assumptions. These generators produce “simulated events�?typically containing:

• Final-state kinematics (momenta, angles, energies)
• Monte Carlo “truth�?labels indicating which fundamental particle or process originated each signature

This data can be used for supervised learning tasks where you already know the truth label. It also forms the basis for unsupervised learning, where you look for patterns among unlabeled data.

3.2 Preprocessing Steps#

1. Filtering: Remove events that don’t meet certain criteria (noise or unphysical states).
2. Normalization: Most ML models perform better if features are on similar scales. For example, energies can range from a few GeV to several TeV, so we often take logarithms or normalize to a standard distribution.
3. Feature Engineering: Derive relevant variables from raw data. For example, you might combine 4-vector momenta into invariant masses or angular separations.

3.3 Storage Formats and Tools#

Particle physics data is often stored in ROOT files (a format developed at CERN). A typical workflow might involve:

�?Converting ROOT to intermediate formats (CSV, HDF5, or parquet files).
�?Using Python libraries like NumPy, pandas, and scikit-hep to manipulate the data.

Below is a snippet showing how you might load ROOT data in Python using uproot, a common library:

1
import uproot
2
import numpy as np
3

4
filename = "example.root"
5
file = uproot.open(filename)
6
tree = file["Events"]
7

8
# Suppose you have branches named px, py, pz, E
9
px = tree["px"].array(library="np")
10
py = tree["py"].array(library="np")
11
pz = tree["pz"].array(library="np")
12
energy = tree["E"].array(library="np")
13

14
# Example: compute invariant mass
15
mass_sq = energy**2 - (px**2 + py**2 + pz**2)
16
mass = np.sqrt(np.where(mass_sq > 0, mass_sq, 0))

4.1 Synthetic Data Generation#

For demonstration, we’ll simulate some synthetic distributions. Imagine we have two event types (A and B), each described by different distributions of energy and momentum.

1
import numpy as np
2
import matplotlib.pyplot as plt
3

4
np.random.seed(42)
5

6
# Generate type A data
7
energy_A = np.random.normal(50, 10, 1000)
8
momentum_A = np.random.normal(20, 5, 1000)
9
labels_A = np.zeros(1000)
10

11
# Generate type B data
12
energy_B = np.random.normal(80, 15, 1000)
13
momentum_B = np.random.normal(10, 3, 1000)
14
labels_B = np.ones(1000)
15

16
# Combine
17
energy = np.concatenate([energy_A, energy_B])
18
momentum = np.concatenate([momentum_A, momentum_B])
19
labels = np.concatenate([labels_A, labels_B])

4.2 Preprocessing#

Let’s scale our data:

1
from sklearn.preprocessing import StandardScaler
2

3
X = np.column_stack((energy, momentum))
4
scaler = StandardScaler()
5
X_scaled = scaler.fit_transform(X)

4.3 Training a Classifier#

We’ll use a simple logistic regression to distinguish Type A from Type B:

1
from sklearn.model_selection import train_test_split
2
from sklearn.linear_model import LogisticRegression
3
from sklearn.metrics import accuracy_score
4

5
X_train, X_test, y_train, y_test = train_test_split(
6
    X_scaled, labels, test_size=0.2, random_state=42
7
)
8

9
model = LogisticRegression()
10
model.fit(X_train, y_train)
11

12
y_pred = model.predict(X_test)
13
acc = accuracy_score(y_test, y_pred)
14
print(f"Accuracy: {acc:.3f}")

The model will likely achieve good accuracy on this synthetic data, demonstrating how easily you can train a basic classifier. In real particle theory problems, the distributions are far more complex, and the training set can be much larger (millions of events).

5.1 Feature Importance#

Techniques such as gradient-based saliency maps or permutation importance can tell us which features (invariant masses, angles, energies) are most influential in distinguishing processes. If a neural network finds that a particular kinematic variable correlates strongly with a new type of event, this could guide further analysis or prompt new theoretical insights.

5.2 Consistency With Known Physics#

Machine Learning might sometimes produce outputs contradictory to well-established physics principles. Ensuring that your pipeline respects basic conservation laws (energy, momentum, charge) can be achieved by embedding physics constraints or designing custom neural network architectures that respect invariants.

5.3 Example: SHAP Values#

SHAP (SHapley Additive exPlanations) is a method to interpret model predictions:

1
import shap
2

3
explainer = shap.Explainer(model, X_train)
4
shap_values = explainer(X_test)
5

6
shap.plots.beeswarm(shap_values)

A beeswarm plot reveals which features drive the classification decisions, allowing you to spot potentially interesting physical variables.

6.1 Dense Neural Networks#

When you have a relatively small set of features—like a handful of kinematic variables—a fully connected (dense) neural network can work well. However, you must be cautious about overfitting. Techniques such as dropout and weight decay help regularize large neural networks.

6.2 Convolutional Neural Networks (CNNs)#

Although CNNs are primarily known for image processing, they can be leveraged in particle physics if your data is converted into an image-like representation. Consider a calorimeter deposit map, where each pixel represents energy deposit in a specific region of the detector. CNNs can capture local correlations akin to image classification tasks.

6.3 Recurrent Neural Networks (RNNs)#

In event-based analysis, you sometimes have sequential data. For instance, you might have a list of particles or time-ordered data from a detector. RNNs or LSTM networks can handle sequences but may be less commonly used in standard collision scenarios compared to other architectures.

6.4 Graph Neural Networks (GNNs)#

A large portion of particle physics data can be modeled as graphs. Jets, for example, can be treated as a set of final-state particles connected by a common origin. GNNs respect the connectivity, allowing the model to learn patterns from adjacency relationships. This has proven to be a powerful approach when analyzing complex, multi-particle signatures.

Below is a simplified example illustrating how you might represent jet data with edges and nodes:

For advanced research, specialized libraries like PyTorch Geometric expedite the creation and training of GNNs for physics data.

8.1 Automated Theory Exploration#

Particle theory is rich with complications, from exploring beyond-the-standard-model (BSM) scenarios to investigating new symmetries. ML can systematically explore large model parameter spaces, identifying regions that produce interesting anomalies or match existing data well.

For example, a technique might combine a genetic algorithm (GA) with an ML surrogate model. The GA navigates the theoretical parameter space, while the ML model quickly predicts the outcome of each set of parameters (e.g., cross sections, decay widths). Promising candidates can then be validated with more detailed computations.

8.2 Symbolic Regression#

Symbolic regression algorithms search for equations that fit data. In particle physics, this can occasionally replace or supplement complicated manual derivations. Tools like PySR or AI Feynman use evolutionary algorithms (or reinforcement learning) to discover physically interpretable formulas. Discovering an elegant closed-form solution for a scattering amplitude, for instance, could save enormous effort and lead to better theoretical understanding.

8.3 Hybrid Approaches#

Ongoing research aims to unite ML with traditional theoretical tools, such as:

• Bayesian inference procedures combined with neural networks to infer posterior distributions of model parameters.
• Surrogate modeling that uses ML to approximate expensive computations like advanced Feynman diagram integrals.