Enter your search into one or more of the boxes below:
You can refine your search by selecting from any of the options below:
Discrete-Time Markov Control Processes: Basic Optimality Criteria

Discrete-Time Markov Control Processes: Basic Optimality Criteria (Hardback)

Printed to order. Despatched in 2-3 weeks.
Email me when back in stock


This book presents the first part of a planned two-volume series devoted to a systematic exposition of some recent developments in the theory of discrete-time Markov control processes (MCPs). Interest is mainly confined to MCPs with Borel state and control (or action) spaces, and possibly unbounded costs and noncompact control constraint sets. MCPs are a class of stochastic control problems, also known as Markov decision processes, controlled Markov processes, or stochastic dynamic pro- grams; sometimes, particularly when the state space is a countable set, they are also called Markov decision (or controlled Markov) chains. Regardless of the name used, MCPs appear in many fields, for example, engineering, economics, operations research, statistics, renewable and nonrenewable re- source management, (control of) epidemics, etc. However, most of the lit- erature (say, at least 90%) is concentrated on MCPs for which (a) the state space is a countable set, and/or (b) the costs-per-stage are bounded, and/or (c) the control constraint sets are compact. But curiously enough, the most widely used control model in engineering and economics--namely the LQ (Linear system/Quadratic cost) model-satisfies none of these conditions. Moreover, when dealing with "partially observable" systems) a standard approach is to transform them into equivalent "completely observable" sys- tems in a larger state space (in fact, a space of probability measures), which is uncountable even if the original state process is finite-valued.

BusinessBusiness & managementOperational researchScience & MathematicsMathematicsApplied mathematicsStochasticsScience & MathematicsMathematicsProbability & statisticsTechnicalMechanical engineering & materialsProduction engineeringIndustrial quality control Publisher: Springer-Verlag New York Inc. Publication Date: 01/12/1995 ISBN-13: 9780387945798  Details: Type: Hardback Format: Books
Availability: Printed to order. Despatched in 2-3 weeks.  

More books by Onesimo Hernandez-Lerma

More books by Jean B. Lasserre

Leave Review


Delivery Options

All delivery times quoted are the average, and cannot be guaranteed. These should be added to the availability message time, to determine when the goods will arrive. During checkout we will give you a cumulative estimated date for delivery.

Location 1st Book Each additional book Average Delivery Time
UK Standard Delivery FREE FREE 3-5 Days
UK First Class £4.50 £1.00 1-2 Days
UK Courier £7.00 £1.00 1-2 Days
Western Europe** Courier £17.00 £3.00 2-3 Days
Western Europe** Airmail £5.00 £1.50 4-14 Days
USA / Canada Courier £20.00 £3.00 2-4 Days
USA / Canada Airmail £7.00 £3.00 4-14 Days
Rest of World Courier £22.50 £3.00 3-6 Days
Rest of World Airmail £8.00 £3.00 7-21 Days

** Includes Austria, Belgium, Denmark, France, Germany, Greece, Iceland, Irish Republic, Italy, Luxembourg, Netherlands, Portugal, Spain, Sweden and Switzerland.

Delivery Help & FAQs

Returns Information

If you are not completely satisfied with your purchase*, you may return it to us in its original condition with in 30 days of receiving your delivery or collection notification email for a refund. Except for damaged items or delivery issues the cost of return postage is borne by the buyer. Your statutory rights are not affected.

* For Exclusions and terms on damaged or delivery issues see Returns Help & FAQs

You might also like

Quantum Fields and Processes: A...
John Gough; Joachim Kupsch
Stochastic Physics and Climate Modelling
Tim Palmer; Paul Williams
Theory of Stochastic Objects:...
Athanasios Christou Micheas
Probability on Graphs: Random...
Geoffrey Grimmett
Latest Blog
On Memory and Forgetting

David Whitehouse talks to us about memory and forgetting, and how his personal experience influenced his new novel, The Long Forgotten.

The End of the F***ing World creator, Charles Forsman, picks his favourite coming of age books

The End of the F***ing World creator, Charles Forsman, picks his favourite coming of age books.

#FoylesFive: Annihilation

With the release of Annihilation on Netflix it's the perfect time to catch up with the book that starts Jeff VanderMeer's unsettling Southern Reach trilogy.

View all Blog Entries
Show/Hide Tweets
© W&G Foyle Ltd