Markovian approximation is a widely-employed idea in the descriptions of the dynamics of open quantum systems. Although it is usually claimed to be a concept inspired by classical Markovianity, the physical meaning of this concept, however, is vague. In this thesis, I compare the descriptions of classical stochastic processes and open quantum systems, and show that those inherent differences lead to the non-trivial problem of characterizing quantum non-Markovianity. With various examples, I argue that the current most often used definitions of quantumMarkovianity in the literature do not fully capture the memoryless property of OQSs. To address this issue, I take a different approach by studying a host of Markov-related concepts in the quantum regime. Some of these concepts have long been used in quantum theory, while others are first proposed in this thesis. I define all these concepts under a unified framework, which allows one to rigorously build hierarchy relations among them. In this sense, quantum non-Markovianity is highly context-dependent. The results in this thesis, summarized as a hierarchy figure, are expected to bring clarity to the nature of quantumnon-Markovianity.