Obesity is a major cancer risk factor, but how differences in systemic metabolism change the tumor microenvironment (TME) and impact anti-tumor immunity is not understood. Here, we demonstrate that high-fat diet (HFD)-induced obesity impairs CD8 T cell function in the murine TME, accelerating tumor growth. We generate a single-cell resolution atlas of cellular metabolism in the TME, detailing how it changes with diet-induced obesity. We find that tumor and CD8 T cells display distinct metabolic adaptations to obesity. Tumor cells increase fat uptake with HFD, whereas tumor-infiltrating CD8 T cells do not. These differential adaptations lead to altered fatty acid partitioning in HFD tumors, impairing CD8 T cell infiltration and function. Blocking metabolic reprogramming by tumor cells in obese mice improves anti-tumor immunity. Analysis of human cancers reveals similar transcriptional changes in CD8 T cell markers, suggesting interventions that exploit metabolism to improve cancer immunotherapy.