Learn Markov chains through working code and model files
Second edition of a programmer-focused Markov chains book, now with C solutions, MATLAB scripts, PRISM models, and spreadsheets for CTMC/DTMC experiments.
- LinkLoot access
- Free
- Provider costs
- Unknown
What you get from it
What it does
This repository holds the supporting materials for Markov Chains for Programmers by Ricardo M. Czekster, second edition published October 2026. The book explains the mathematical foundations of Markov chains and basic solution methods, aimed at programmers who want to understand how these models work rather than just apply them.
The repo provides C programming code for every challenge in the book, MATLAB scripts for selected solutions, PRISM model-checker inputs for both CTMC and DTMC examples, and MS-Excel spreadsheets that reproduce the experiments. A collection of ready-to-run textual model files covers classic examples: the Pushkin model (the original Markov weather example), Land of Oz, Belfast weather, software aging and rejuvenation, and a lighting control model. Each file is named by state count and type, making it easy to find what you need.
Two project folders organize the C code differently. project00 gives each challenge its own independent main() function with a shared Makefile and auxiliary functions in markov.c. project01 consolidates everything into a single command-line tool with built-in options, described in Challenge 07.
Who it helps
Developers, students, and self-taught programmers who encounter Markov chains in performance evaluation, reliability modeling, or stochastic simulation and want concrete code to study alongside theory. The companion pre-print on Performance Evaluation links to ResearchGate for additional context. No prior probability background is assumed beyond comfort with reading code.
Getting started
Clone the repository from GitHub. The PDF version of the book is linked directly from the README via a figshare DOI. Start with the challenges folder to see what problems are posed, then look at the corresponding C source in project00 or project01. Model files in models/ can be run internally or loaded into PRISM (free download available at prismmodelchecker.org). MATLAB scripts require MATLAB or Octave; spreadsheet files need Excel or LibreOffice Calc.
Limits and costs
The book text is licensed CC BY 4.0; all code is GPLv3. Hosting and access are free. The author notes LLM-assisted passages in the second edition are marked in grey but does not specify which sections those are. Some advanced topics like uniformised and embedded DTMCs were added in this edition; coverage depth is unclear without reading the full text. The repository does not include build instructions beyond the Makefile in project00, so Windows users may need WSL or a ported toolchain.
Source links
Discussion
Share practical experience, questions, or warnings with the community.
Sign in to join the discussion and vote on comments.
Sign in