Immunity to chronic viral infections and tumors is tightly associated with ‘exhausted’ T cells, which display loss of effector function and expression of inhibitory receptors, most notably PD-1. Exhausted T cell responses are sustained over long periods by a subset of specialized T cells that retain high proliferative capacity and the ability to self-renew while also displaying hallmarks of T cell exhaustion. Importantly, these ‘exhausted memory’ T cells or TMEX are responsible for the boost of immunity following checkpoint blockade in immunotherapy. Here we show that during the onset of an immune response to a chronic infection, early memory but not effector T cells acquire features of exhaustion, including an epigenetic and transcriptional profile of exhausted T cells. Importantly, this early TMEX population propagates this differentiation pattern by replenishing the effector response with exhausted T cells in a process that depends on the expression of the transcriptional regulator ID3. Overall, we demonstrate that T cell exhaustion is manifested early during T cell activation and show that a specialized population of memory T cells initially acquires and propagates this differentiation pattern.