CO2 has not been the primary driver of temperature through time
Earth's temperature has cycled between very hot climates (hothouse) and very cold climates (icehouse) like we are experiencing right now. Both of these occurred during periods of both high and low CO2.
Carbon Dioxide Reconstruction: Berner RA, Kothavala Z (2001) GEOCARB III: A revised model of atmospheric CO2 over Phanerozoic time, IGBP PAGES and World Data Center for Paleoclimatology, Data Contribution Series # 2002-051. NOAA/NGDC Paleoclimatology Program, Boulder CO, USA.
Temperature Reconstruction: Scotese CR (2002) Analysis of the temperature oscillations in geological eras. Paleomap Project