Author: Ernad Mujakic
Date: 2025-07-02

Probability is a branch of mathematics that deals with the analysis of events and their likelihood of occurring. The probability of an event, written is a number between 0 and 1, with 0 representing impossible and 1 representing certainty. Probability theory is a framework for making inferences of events that have elements of uncertainty or randomness within them or their outcomes.

Random Experiment - any process or action that yields uncertain outcomes.

Sample Space - commonly denoted as , is the set of all possible outcomes of a random experiment.

Event - a subset of the sample space, representing specific outcomes.

Power Set: The set of all possible subsets of a sample space, including the empty set or .

The probability of an event is calculated by dividing the number of favorable outcomes by the total size of the sample space:

Joint probability is the probability of 2 events happening simultaneously. If 2 events are independent, then their joint probability is:

If 2 events are mutually exclusive, then the probability of one or the other occurring is equal to the sum of their probabilities:

If 2 events aren't necessarily mutually exclusive, then you simply subtract their intersection in order to prevent counting the values in their intersection twice:

The probability of one event occurring, given that another has already occurred, denoted as (the probability of A given B), is:

Bayes' Theorem states that:

The factorial of a non-negative integer , denoted , is the product of all positive integers less than or equal to :

A permutation of a set is a possible arrangement of it's elements, where the order of the elements matters. The number of possible permutations of elements from a set with total elements is given by:

A combination of a set is a selection of it's elements where the order does not matter. The number of combinations of objects from a set with total elements is given by:

A random variable is a formalization that assigns a numerical value to the outcome of a random event or experiment. Random variables can be classified into two main types:

Random variables are typically denoted by capital letters. For example, the set can represent the random variables 'heads' and 'tails' in a coin flip experiment. Their values are represented with their corresponding lowercase letter.

A probability distribution is a mathematical function which describes the likelihood of different outcomes for the domain of a random variable. The probability of each outcome is between 0 and 1 (inclusive), and the sum of probabilities of each outcome must sum to 1.

Discrete probability distributions describe the probability of each possible value in the domain of a discrete random variable.

The Probability Mass Function (PMF) gives the probability of each possible value of a random variable :

The sum of all outcomes must sum to 1:

Common Discrete Distributions:

Continuous probability distributions describe the probability of a continuous random variable. These distributions are characterized by a Probability Density Function (PDF).

The probability density function describes the likelihood of a random variable taking on a value in a specific range:

where is the PDF and must satisfy:

Common Continuous Distributions:

The expected value, or mean, of a random variable , denoted , is a measure of central tendency that represents the average outcome of a random variable. This concept is crucial in various fields, including artificial intelligence, especially in the context of stochastic (random) task environments, where uncertainty plays a significant role.

For discrete probability distributions, the expected value is defined as:

Where is the Probability Mass Function.

For continuous probability distributions, the expected value is defined as:

Where is the Probability Density Function.

Probability is the foundation of many Machine Learning algorithms such as:

Decision theory utilizes probability to make rational decisions under uncertainty. Relevant subject include:

A Bayesian Network is a probabilistic model implemented as a directed acyclic graph that represents a set of random variables and their conditional dependencies, as well as a set of conditional probability distribution tables.

Game theory is a mathematical framework for modelling strategic interactions in a multi-agent interdependent environment. Probability is crucial for game theory, where agents face uncertainty over the actions of other players as well as the state of the game.

A Markov process is a mathematical model which models a stochastic environment that satisfies the Markov Property. The Markov property states that the future state depends only on the present state and not past states. Markov processes have transition probabilities between states, quantifying the chance of moving from one state to another.

Bayesian statistics is subfield of Statistics that uses Bayes' theorem to update the probability of a hypothesis as new evidence becomes available. Bayesian methods are widely used in classification, regression, and decision-making algorithms.