Uncertain Reasoning - Probabilistic AI

Why Uncertainty in AI?

Classical logic assumes complete, certain knowledge: every statement is definitively true or false. But the real world is full of uncertainty—medical tests have false positives, weather forecasts are probabilistic, and sensor readings are noisy. An AI system that demands certainty before acting would be paralyzed by the complexity of real-world decision-making.

Uncertain reasoning provides formal frameworks for making rational decisions despite incomplete, ambiguous, or probabilistic information. These frameworks determine how to combine uncertain evidence, update beliefs as new information arrives, and choose actions that maximize expected utility.

Sources of Uncertainty

Probability Theory: The Foundation

Bayes' Theorem

Medical Diagnosis Example

Bayesian Networks

Structure

A Bayesian network is a directed acyclic graph where:

• Nodes represent random variables
• Edges represent direct probabilistic dependencies
• Each node stores conditional probability tables (CPTs)

Inference in Bayesian Networks

Certainty Factors (MYCIN Approach)

MYCIN used certainty factors (CF) instead of probability:

Dempster-Shafer Theory

Handles ignorance explicitly (unlike probability which assigns it implicitly):

Fuzzy Logic for Uncertainty

Decision Theory: Acting Under Uncertainty

Comparison of Uncertainty Methods

Interview Questions

Q: Why not just use probability for everything? A: Probability struggles with ignorance vs. disbelief. P(A) = 0.5 could mean "equally likely true/false" (coin flip) or "I have no idea" (complete ignorance). Dempster-Shafer distinguishes these. Also, acquiring precise probabilities from experts is difficult—certainty factors and fuzzy values are more elicitable.

Q: What is the difference between a Bayesian network and a neural network? A: Bayesian networks encode explicit causal/probabilistic relationships (interpretable structure). Neural networks learn implicit patterns from data (black box). BNs require expert-designed structure + probability tables; NNs require only data. BNs give calibrated uncertainty; NNs need special techniques (dropout, ensembles) for uncertainty.

Q: How do you choose between uncertainty frameworks? A: Use probability/Bayesian networks when you have good data and need calibrated predictions. Use certainty factors when building expert systems with human-elicited confidence. Use Dempster-Shafer when distinguishing ignorance matters (sensor fusion). Use fuzzy logic when concepts are inherently vague rather than random.