Abstract

Dynamic systems described by differential equations often involve feed

back among system components. When there are time delays for components to

sense and respond to feedback, delay differential equation (DDE) models are

commonly used. This paper considers the problem of inferring unknown system

parameters, including the time delays, from noisy and sparse experimental data

observed from the system.

We propose an extension of manifold-constrained

Gaussian processes to conduct parameter inference for DDEs, whereas the time

delay parameters have posed a challenge for existing methods that bypass numerical solvers. Our method uses a Bayesian framework to impose a Gaussian process

model over the system trajectory, conditioned on the manifold constraint that

satisfies the DDEs. For efficient computation, a linear interpolation scheme is

developed to approximate the values of the time-delayed system outputs, along

with corresponding theoretical error bounds on the approximated derivatives.

Two simulation examples, based on Hutchinson’s equation and the lac operon

system, together with a real-world application using Ontario COVID-19 data,

are used to illustrate the efficacy of our method.

Information

Preprint No.SS-2024-0213
Manuscript IDSS-2024-0213
Complete AuthorsYuxuan Zhao, Samuel W.K. Wong
Corresponding AuthorsSamuel W.K. Wong
Emailssamuel.wong@uwaterloo.ca

References

  1. Bellen, A. and M. Zennaro (2013). Numerical methods for delay differential equations. Numerical Mathematics and Scientific Computation.
  2. Bihorel, S. (2011). The scaRabee Package: An R-based Tool for Model Simulation and Optimization in Pharmacometrics. https://cran.r-project.org/package=scaRabee.
  3. Boersch-Supan, P. H., S. J. Ryan, and L. R. Johnson (2017). deBInfer: Bayesian inference for dynamical models of biological systems in R. Methods in Ecology and Evolution 8(4), 511–518.
  4. Calderhead, B., M. Girolami, and N. Lawrence (2008). Accelerating Bayesian inference over nonlinear differential equations with Gaussian processes. Advances in neural information processing systems 21, 217–224.
  5. Dondelinger, F., D. Husmeier, S. Rogers, and M. Filippone (2013). ODE parameter inference using adaptive gradient matching with Gaussian processes. In Artificial intelligence and statistics, pp. 216–228. PMLR.
  6. Ellner, S. P., B. E. Kendall, S. N. Wood, E. McCauley, and C. J. Briggs (1997). Inferring mechanism from time-series data: delay-differential equations. Physica D: Nonlinear Phenomena 110(3-4), 182–194.
  7. Erneux, T. (2009). Applied delay differential equations. Springer.
  8. Glass, D. S., X. Jin, and I. H. Riedel-Kruse (2021). Nonlinear delay differential equations and their application to modeling biological network motifs. Nat. Commun. 12(1), article 1788.
  9. Horbelt, W., J. Timmer, and H. U. Voss (2002). Parameter estimation in nonlinear delayed feedback systems from noisy data. Physics Letters A 299(5-6), 513–521.
  10. Hutchinson, G. E. (1948). Circular causal systems in ecology. Annals of the New York Academy of Sciences 50(4), 221–246.
  11. Karniadakis, G. E., I. G. Kevrekidis, L. Lu, P. Perdikaris, S. Wang, and L. Yang (2021). Physics-informed machine learning. Nature Reviews Physics 3(6), 422–440.
  12. Kuang, Y. (1993). Delay differential equations: with applications in population dynamics. Academic press.
  13. Liang, H. and H. Wu (2008). Parameter estimation for differential equation models using a framework of measurement error in regression models. Journal of the American Statistical Association 103(484), 1570–1583.
  14. Liu, C. and F. Wang (2020). Parameter identification of genetic regulatory network with time-varying delays via adaptive synchronization method. Advances in Difference Equations 2020, article 127.
  15. Ma, W., M. Song, and Y. Takeuchi (2004). Global stability of an SIR epidemic model with time delay. Applied Mathematics Letters 17(10), 1141–1145.
  16. May, R. M. (1976). Simple mathematical models with very complicated dynamics. Nature 261, 459–467.
  17. Neal, R. M. (2011). MCMC Using Hamiltonian Dynamics. In Handbook of Markov Chain Monte Carlo, pp. 113–162. Chapman and Hall/CRC.
  18. Nicholson, A. J. (1954). Compensatory reactions of populations to stresses, and their evolutionary significance. Australian Journal of Zoology 2(1), 1–8.
  19. Pardee, A. B., F. Jacob, and J. Monod (1959). The genetic control and cytoplasmic expression of “inducibility” in the synthesis of β-galactosidase by E. coli. Journal of Molecular Biology 1(2), 165–178.
  20. Pei, L. and D. Liu (2023). Long-term prediction of the COVID-19 epidemics induced by Omicron-virus in China based on a novel non-autonomous delayed SIR model. Journal of Physics A: Mathematical and Theoretical 56(23), 235601.
  21. Ramsay, J. O., G. Hooker, D. Campbell, and J. Cao (2007). Parameter estimation for differential equations: a generalized smoothing approach. Journal of the Royal Statistical Society Series B: Statistical Methodology 69(5), 741–796.
  22. Shao, J., R. Fan, J. Hu, T. Zhang, C. Lee, X. Huang, F. Wang, H. Liang, Y. Jin, Y. Jiang, et al.
  23. (2022). Clinical progression and outcome of hospitalized patients infected with SARSCoV-2 omicron variant in Shanghai, China. Vaccines 10(9), article 1409.
  24. Soetaert, K., T. Petzoldt, and R. W. Setzer (2010). Solving differential equations in R: package deSolve. Journal of statistical software 33, 1–25.
  25. Tronarp, F., N. Bosch, and P. Hennig (2022). Fenrir: Physics-enhanced regression for initial value problems. In International Conference on Machine Learning, pp. 21776–21794. PMLR.
  26. Varah, J. M. (1982). A spline least squares method for numerical parameter estimation in differential equations. SIAM Journal on Scientific and Statistical Computing 3(1), 28–46.
  27. Wang, L. and J. Cao (2012). Estimating parameters in delay differential equation models. Journal of agricultural, biological, and environmental statistics 17, 68–83.
  28. Wang, S., S. Ge, R. Doig, and L. Wang (2022). Adaptive semiparametric Bayesian differential equations via sequential Monte Carlo. Journal of Computational and Graphical Statistics 31(2), 600–613.
  29. Wang, W., R. Tuo, and C. Jeff Wu (2020). On prediction properties of kriging: Uniform error bounds and robustness. Journal of the American Statistical Association 115(530), 920–930.
  30. Wenk, P., A. Gotovos, S. Bauer, N. S. Gorbach, A. Krause, and J. M. Buhmann (2019). Fast Gaussian process based gradient matching for parameter identification in systems of nonlinear ODEs. In The 22nd International Conference on Artificial Intelligence and Statistics, pp. 1351–1360. PMLR.
  31. Wise, J. (2022). Covid-19: Symptomatic infection with omicron variant is milder and shorter than with delta, study reports. BMJ 377, o922.
  32. Wong, S. W., S. Yang, and S. Kou (2023). Estimating and Assessing Differential Equation Models with Time-Course Data. The Journal of Physical Chemistry B 127(11), 2362– 2374.
  33. Wong, S. W. K., S. Yang, and S. C. Kou (2024). magi: A package for inference of dynamic systems from noisy and sparse data via manifold-constrained Gaussian processes. Journal of Statistical Software 109(4), 1–47.
  34. Wu, M. and M. Lysy (2024). Data-Adaptive Probabilistic Likelihood Approximation for Ordinary Differential Equations. In International Conference on Artificial Intelligence and Statistics, pp. 1018–1026. PMLR.
  35. Yang, L., X. Meng, and G. E. Karniadakis (2021). B-PINNs: Bayesian physics-informed neural networks for forward and inverse PDE problems with noisy data. Journal of Computational Physics 425, 109913.
  36. Yang, S., S. W. Wong, and S. Kou (2021). Inference of dynamic systems from noisy and sparse data via manifold-constrained Gaussian processes. Proceedings of the National Academy of Sciences 118(15), e2020397118.
  37. Yildirim, N. and M. C. Mackey (2003). Feedback regulation in the lactose operon: a mathematical modeling study and comparison with experimental data. Biophysical journal 84(5), 2841–2851.

Acknowledgments

This work was partially supported by Discovery Grant RGPIN-2019-04771

from the Natural Sciences and Engineering Research Council of Canada.

Supplementary Materials

Includes the proofs, Tables, and Figures referenced in Sections 1,3,4,5.

Supplementary materials are available for download.