In recent years, there has been a rising demand for transparency and explainability in artificial intelligence. Although significant progress has been made in generating different types of explanations for machine learning models, this topic has received minimal attention in the operations research community, due to a larger focus by the public on societal effects of data-driven machine learning models. However, algorithmic decisions in operations research are made by complex algorithms, which also lack explainability. The main goal of this PhD project is to build the foundation for explainable decision making. The research will focus on defining a general mathematical framework for explainable decision making and introducing models to provide explanations for different classes of optimization problems. Besides, solution algorithms will be developed and tested on real-world instances of operations research problems.