Autoencoder-Based Likelihood-Free Parameter Inference of Gene Regulatory Network

Paper Info

Title	Autoencoder-Based Likelihood-Free Parameter Inference of Gene Regulatory Network
Author	Liang Cheng
Master Thesis	DiVA: diva2:1777819
Code	GitHub: autoencoder-as-summary-statistics-for-likelihood-free-parameter-inference

---

Motivation

Gene regulatory networks (GRNs) are fundamental to understanding how cells function. A key question is: given observed time-series data of gene expression, can we infer the underlying kinetic parameters that govern the network?

Traditional Bayesian inference requires computing the likelihood $p(x|\theta)$ — the probability of observing data $x$ given parameters $\theta$. But for GRNs modeled with stochastic simulators (e.g., Gillespie algorithm), the likelihood is intractable: we can simulate data, but we can’t compute its probability analytically.

This is where Likelihood-Free Inference (LFI) comes in. LFI methods like Approximate Bayesian Computation (ABC) and Neural Posterior Estimation (NPE) work by comparing simulated data to observed data using summary statistics — hand-crafted features that capture the essence of the data.

The problem: Choosing good summary statistics is hard, especially for time-series data. Poor summary statistics throw away information and lead to inaccurate posterior estimates.

Our idea: Instead of hand-designing summary statistics, let an autoencoder learn them automatically. The autoencoder’s latent space becomes a compressed, information-preserving representation of the time-series data — exactly what we need for LFI.

---

Pipeline Overview

     θ (parameters: 15-dim)
     │
     ▼
┌──────────────────────────┐
│  Gillespie Simulator      │  ← Vilar genetic oscillator
│  (Vilar GRN model)        │
└────────────┬─────────────┘
             │
             │ x (time series: 3 species × 201 time steps)
             ▼
┌──────────────────────────┐
│   Noise Conditional       │
│   Autoencoder (ENCA)      │  ← learns compressed representation
│                           │
│  Encoder → z (16-dim) →   │
│  Decoder → x̂              │
└────────────┬─────────────┘
             │
             │ z = summary statistics
             ▼
┌──────────────────────────┐
│   SNPE (Sequential Neural │
│   Posterior Estimation)   │  ← infers p(θ|z_obs)
└────────────┬─────────────┘
             │
             ▼
     Posterior distribution
     p(θ | x_obs)

---

Method Details

1. Task Setup: Vilar Genetic Oscillator

We study a well-known Vilar oscillator — a gene regulatory network with 3 species (mRNA, protein A, protein R) governed by 15 kinetic parameters.

Item	Details
Model	Vilar genetic oscillator (stochastic GRN)
Simulator	Gillespie algorithm (exact stochastic simulation)
Observables	3 molecular species × 201 time steps → 603 values per trajectory
Parameters	15 kinetic parameters to infer
Data	10,000 simulated trajectories (8,000 train / 2,000 test)

2. Summary Statistics: The Core Problem

The quality of likelihood-free inference depends entirely on the summary statistics $S(x)$:

• Hand-crafted statistics: mean, std, autocorrelation, FFT features… domain expertise required, may discard critical information
• Raw data: too high-dimensional (603-D) → curse of dimensionality, inefficient for ABC/NPE

Our approach: Use an autoencoder to learn a low-dimensional latent representation $z = \text{Encoder}(x)$ that preserves the information needed for parameter inference.

3. Explicit Noise Conditional Autoencoder (ENCA)

We design a special autoencoder variant called ENCA (Explicit Noise Conditional Autoencoder):

Component	Details
Architecture	Symmetric encoder-decoder
Input	Time series of 3 species × 201 time steps
Latent dimension	16 (much smaller than 603 raw dimensions)
Noise conditioning	Explicit noise level injected during training for robustness
Loss	Reconstruction loss (MSE between input and reconstructed output)

Why ENCA? Stochastic simulation produces noisy trajectories — multiple runs with the same $\theta$ look different. The noise conditioning helps the autoencoder learn to extract the signal (the underlying dynamics determined by $\theta$) from the noise (stochastic fluctuations).

4. Autoencoder Architectures Compared

We test four different autoencoder architectures to see which learns the best summary statistics:

Architecture	Strength	Rationale
FNN	Simple baseline	Treats data as flat vector
LSTM	Sequential modeling	Standard for time series
CNN	Local pattern detection	Captures temporal motifs
TCN (Temporal Convolutional Network)	Long-range dependencies	Dilated convolutions for time series

5. LFI with SNPE

Once we have the autoencoder, we use its encoder to convert time series into 16-D latent vectors, then run SNPE (Sequential Neural Posterior Estimation):

Round 1: sample θ ~ prior → simulate x → encode z → train neural density estimator
    │
    ▼
Round 2: sample θ ~ q(θ|z_obs) → simulate x → encode z → refine estimator
    │
    ▼
Round N: final posterior estimate q(θ|z_obs) ≈ p(θ|x_obs)

Baselines compared:

• Traditional CNN features: hand-designed feature extraction
• ABC-SMC: Approximate Bayesian Computation with Sequential Monte Carlo (planned but encountered library conflicts)

---

Results

1. Autoencoder Reconstruction Quality

Architecture	Reconstruction Error	Quality
FNN	Highest	Poor — treats time series as unstructured vector
LSTM	Medium	Decent — captures sequential patterns but struggles with fine details
CNN	Lowest	Best — captures local temporal patterns effectively
TCN	Medium-High	Good but underperforms CNN on this task

Surprising finding: CNN outperforms LSTM and TCN on this GRN reconstruction task. The local temporal features (peaks, troughs, oscillation periods) are more important than long-range sequential dependencies for this particular system.

2. Parameter Inference Performance

We compare inference quality using the autoencoder’s latent space as summary statistics vs. traditional CNN feature extraction:

Method	Inference Quality	Notes
Traditional CNN features	Baseline	Hand-designed features
FNN autoencoder	Limited	Poor reconstruction → poor summary statistics
LSTM autoencoder	Moderate	Better than FNN, but not optimal
CNN autoencoder	Best	Learned features outperform hand-designed ones
TCN autoencoder	Good	Below CNN on most parameters

3. Key Findings

1. Learned summary statistics beat hand-crafted ones — CNN autoencoder latent space consistently produces better parameter estimates than traditional feature extraction
2. CNN is the best autoencoder architecture for this task — despite LSTM/TCN being designed for sequences, local temporal patterns matter more for GRN inference
3. Dimensionality reduction works — going from 603-D raw data to 16-D latent space preserves enough information for accurate parameter inference
4. Parameters with smaller numerical ranges are harder to infer — the method shows varying accuracy across the 15 parameters

---

Discussion & Limitations

Aspect	Discussion
ABC-SMC not implemented	PyTorch multiprocessing conflicts with ABC library prevented ABC-SMC comparison; SNPE was used as the primary inference method instead
One test system	Results are on the Vilar oscillator only; generalizability to other GRN topologies needs verification
Latent dimension choice	16-D was chosen empirically; systematic study of dimensionality vs. inference quality could be valuable
Information-theoretic guarantee	Autoencoder optimizes reconstruction loss, not information about $\theta$ — information-preserving but not necessarily information-optimal for inference
Future: contrastive learning	Could train the encoder directly to maximize mutual information with $\theta$ (e.g., via contrastive learning or InfoNCE loss) instead of reconstruction
Future: larger networks	More complex GRNs with more species and parameters would be a more challenging test
Future: real experimental data	All experiments are on simulated data; validation on real biological data remains an open challenge

---

Key Takeaways

1. Autoencoders can learn effective summary statistics for likelihood-free inference — no hand-engineering required
2. CNN autoencoders work surprisingly well for time series — don’t assume LSTM/TCN are always better for sequential data
3. The encoder latent space is information-preserving — a 16-D vector from a 603-D input still captures enough signal for parameter inference
4. Explicit noise conditioning helps — ENCA makes the learned representation robust to stochastic simulation noise
5. A practical pipeline for GRN inference — autoencoder + SNPE provides an end-to-end approach for inferring gene regulatory network parameters

One-sentence summary: A noise-conditional CNN autoencoder learns compressed, information-preserving summary statistics from stochastic gene expression time series, enabling accurate likelihood-free parameter inference of gene regulatory networks via sequential neural posterior estimation.

---

BibTeX

@mastersthesis{cheng2023autoencoder,
  title={Autoencoder-Based Likelihood-Free Parameter Inference of Gene Regulatory Network},
  author={Cheng, Liang},
  year={2023},
  school={Uppsala University},
  type={Master's Thesis in Data Science (30 credits)},
  url={https://uu.diva-portal.org/smash/record.jsf?pid=diva2%3A1777819}
}