1. Start with an unobserved state
A Markov-switching model represents observations using an unobserved state that can change over time. In a first-order formulation, the next state's distribution depends on the current state and any specified transition covariates, rather than the entire state history independently. Statsmodels documents regression specifications in which selected coefficients or variances can differ across states. The state is a modeling device, not an observed economic fact. This reference explains a method; it does not claim that an accompanying threshold demonstration trains such a model or that these models provide investment advice.
2. Separate transitions from observation likelihoods
Transition probabilities describe movement between latent states. The observation model describes how a measured outcome can arise within each state. Filtering combines the transition-based prior with the likelihood of the newly observed data, then normalizes across states. A state can become more plausible because the new observation fits it better, even without a dramatic change in the measured series. Document which parameters switch and which remain shared. Otherwise, a label interpreted as high volatility might actually distinguish fitted means, regression sensitivities, or a combination of features.
3. Identify the probability's information set
A predicted probability for period t conditions on observations through t-1. A filtered probability includes the observation at t. A smoothed probability for t also uses observations after t, up to the specified sample endpoint. Statsmodels exposes these distinct probability choices. Smoothing is appropriate for retrospective inference, but its improved historical clarity cannot be credited to an earlier decision maker. Availability adds another condition: the observation indexed t must actually have been released before the decision deadline. A time-series index by itself does not establish that eligibility.
4. Audit parameter estimation as well as filtering
A probability labeled filtered can still contain hindsight if the model parameters were estimated using later observations. Reconstructing a decision-time experiment therefore requires eligible data vintages, a training endpoint no later than the decision, and an explicit refit schedule. Record the initialization and estimation settings too. Statsmodels' Markov-autoregression example notes that likelihood optimization can encounter local maxima. Comparing several starting values can diagnose optimization sensitivity; it does not create independent economic evidence. A converged fit establishes a numerical status, not the truth or usefulness of its state interpretation.
5. Read persistence without turning it into a promise
A transition matrix needs an orientation convention: identify whether rows or columns represent the origin state, and check that each origin's destination probabilities sum to one. Under a time-homogeneous Markov model, persistence implies a model-based duration distribution, not a scheduled end date for the current state. Time-varying transition specifications require additional care. Sparse transitions and overlapping state distributions can make estimates difficult to distinguish. Report estimation uncertainty and specification sensitivity separately from the conditional state probabilities, which usually presume the fitted parameter values and chosen model structure.
6. Compare fictional information cutoffs
The fictional table shows which observations enter three calculations for one target period. It intentionally supplies no invented probability estimates. Hold the parameter version fixed when explaining the observation-set difference, then separately ask whether that parameter version existed at the historical deadline. A smoothed path can disagree with a filtered path without either calculation being defective: they answer different conditional questions. If a chart converts probabilities into one dominant-state label, preserve the underlying vector and explain the selection rule, including ties and any abstention threshold.
7. Require evidence beyond an attractive fitted path
Inspect convergence diagnostics, state occupancy, parameter sensitivity, residual behavior, and stability under reasonable alternative specifications. State numbers are identifiers, so align their interpretations before comparing separate fits. Keep descriptive fit quality distinct from out-of-sample predictive assessment and from the performance of any downstream decision rule. Probabilities conditional on a fitted model do not automatically measure its calibration against an independently observable target. A responsible conclusion can be that a model summarizes a sample coherently while real-time usefulness remains unresolved. Do not let confident-looking curves conceal that evidence boundary.
Inspect the comparison
| Fictional calculation for period t | Observations used | Interpretation limit |
|---|---|---|
| Predicted | Through t-1 | Before observing the target-period measurement |
| Filtered | Through t | Still requires an eligible historical parameter version |
| Smoothed | Through later endpoint T | Retrospective inference using later observations |
Primary references
References support the stated methods and data conventions. Original fictional examples are not observations from the cited sources.