Markov Decision Processes for sequential decision problems

[Home]

Time schedule

Wednesday, July 23rd 2014

Software

Software included in the Advanced Herd Management package will be used.

Literature

Theory

• Jensen, F.V. 2001. Bayesian Networks and Decision Graphs, Springer, 2001. Chapters 1, 2, 7.6.

• Jørgensen, E., A.R. Kristensen & D. Nilsson. 2014. Markov LIMID processes for representing and solving renewal problems. Annals of Operations Research 219, 63-84.

• Kristensen, A.R. 2003. A general software system for Markov decision processes in herd management applications. Computers and Electronics in Agriculture 38, 199-215.
• Kristensen, A.R. & E. Jøgensen. 2000. Multi-level hierarchic Markov processes as a framework for herd management support. Annals of Operations Research 94, 69-89.
• Kristensen, A.R., E. Jøgensen & N. Toft. 2010. Herd Management Science. University of Copenhagen. Chapters 9, 12, 13.
• Lauritzen, S.L. & D. Nilsson. 2001. Representing and Solving Decision Problems with Limited Information. Management Science 47, 1235-1251.
• Nielsen, L.R. E. Jøgensen & S. Højsgaard. 2011. Embedding a state space model into a Markov decision process. Annals of Operations Research 190, 289-309.
• Nielsen, L. R., Kristensen, A. R., 2014. Markov decision processes to model livestock systems. In: Plà-Aragonés, L. M. (Ed.), Handbook of Operations Research in Agriculture and the Agri-Food Industry. Springer. In press.

Applications with Bayesian updating

• Demeter, R.M., A.R. Kristensen, J. Dijkstra, A.G.J.M. Oude Lansink, M.P.M. Meuwissen & J.A.M. van Arendonk. 2011. A multi-level hierarchic Markov process with Bayesian updating for herd optimization and simulation in dairy cattle. Journal of Dairy Science 94, 5938-5962.
• Ge, L., A.R. Kristensen, M.C.M. Mourits & R.B. Huirne. 2014. A new decision support framework for managing foot-and-mouth disease epidemics. Annals of Operations Research 219, 49-62.
• Ge, L., M.C.M. Mourits, A.R. Kristensen & R.B.M. Huirne. 2010. A modelling approach to support dynamic decision-making in the control of FMD epidemics. Preventive Veterinary Medicine 95, 167-174.
• Lien, G., A.R. Kristensen, A. Hegrenes & J.B. Hardaker. 2003. Optimal length of leys in an area with winter damage problems. Grass and Forage Science 58, 168-177.
• Kristensen, A.R. & T.A. Søllested. 2004. A sow replacement model using Bayesian updating in a 3-level hierarchic Markov process. I. Biological model. Livestock Production Science 87, 13-24.
• Kristensen, A.R. & T.A. Søllested. 2004. A sow replacement model using Bayesian updating in a three-level hierarchic Markov process II. Optimization model. Livestock Production Science 87, 25-36.
• Nielsen, L.R., E. Jørgensen, A. R. Kristensen, and S. Østergaard. 2010. Optimal replacement policies for dairy cows based on daily yield measurements. Journal of Dairy Science 93, 75-92.
• Rodríguez, S.V., T.B. Jensen, L.M. Plà & A.R. Kristensen. 2011. Optimal replacement policies and economic value of clinical observations in sow herds. Livestock Science 138, 207-219.
• Verstegen, J.A.A.M., J. Sonnemans, R.B.M. Huirne, A.A. Dijkhuizen & J.C. Cox, James. 1998. Quantifying the effects of sow-herd management information systems on farmers' decision making using experimental economics. American Journal of Agricultural Economics 80, 821.

Other applications based on hierarchical models

• Bar, D., L. W. Tauer, G. Bennett, R. N. González, J. A. Hertl, Y. H. Schukken, H. F. Schulte, F. L. Welcome & Y. T. Gröhn. 2008. The Cost of Generic Clinical Mastitis in Dairy Cows as Estimated by Using Dynamic Programming. Journal of Dairy Science 91, 2205–2214.
• Bar, D., L.W. Tauer, G. Bennett, R.N. González, J.A. Hertl, H.F. Schulte, Y.H. Schukken, F.L. Welcome & Y.T. Gröhn. 2009. Use of a dynamic programming model to estimate the value of clinical mastitis treatment and prevention options utilized by dairy producers. Agricultural Systems 99, 6-12
• Cha, E., J.A. Hertl, D. Bar & Y.T. Gröhn. 2010. The cost of different types of lameness in dairy cows calculated by dynamic programming. Preventive Veterinary Medicine 97, 1-8.
• Cha, E., A.R. Kristensen, J.A. Hertl, Y.H. Schukken, L.W. Tauer, F.L. Welcome, & Y.T. Gröhn. 2014. Optimal insemination and replacement decisions to minimize the cost of pathogen specific clinical mastitis in dairy cows. Journal of Dairy Science 97, 2102-2117. [Full text]
• Haran, P. 1997. Markov decision processes in the optimisation of culling decisions for Irish dairy herds. Master's thesis. School of Computer Applications, Dublin City University.
• Houben, E. P. H., R. B. M. Huirne, A. A. Dijkhuizen & A. R. Kristensen. 1994. Optimal replacement of mastitis cows determined by a hierarchic Markov process. Journal of Dairy Science 77, 2975-2993. [Full text]
• Kristensen, A. R. 1987. Optimal replacement and ranking of dairy cows determined by a hierarchic Markov process. Livestock Production Science 16, 131-144. [Full text]
• Kristensen, A. R. 1989. Optimal replacement and ranking of dairy cows under milk quotas. Acta Agriculturæ Scandinavica 39, 311-318. [Full text]
• Mourits, M.C.M., R.B.M. Huirne, A.A. Dijkhuizen, A.R. Kristensen & D.T. Galligan. 1999. Economic optimization of dairy heifer management decisions. Agricultural Systems 61, 17-31.  [Full text]
• Nielsen, B.K., A.R. Kristensen & S.M. Thamsborg. 2004. Optimal decisions in organic steer production - a model including winter feeding level, grazing strategy and slaughtering policy. Livestock Production Science 88, 239-250.
• Nielsen, B.K. & A.R. Kristensen. 2007. Optimal decisions in organic beef production from steers – effects of criterion of optimality and price changes. Livestock Science 110, 25-32.
• Stygar, A.H., A.R. Kristensen & J. Makulska. Optimal management of replacement heifers in a beef herd: A model for simultaneous optimization of rearing and breeding decisions. Journal of Animal Science. In press.
• Toft, N., A.R. Kristensen & E. Jørgensen. 2005. A framework for decision support related to infectious diseases in slaughter pig fattening units. Agricultural Systems 85, 120-137.