The coronavirus disease 2019 (COVID-19) pandemic has disrupted childhood vaccinations, including pneumococcal conjugate vaccine (PCV). Evaluating the possible impact on the invasive pneumococcal disease (IPD) incidence associated with a decline in childhood pneumococcal vaccination is important to advocate the PCV programs. Using a deterministic, dynamic transmission model, the differential incidence and burden of IPD in children younger than 5 years in Japan were estimated between the rapid vaccination recovery (January 2021) and the delayed vaccination recovery (April 2022) scenarios for the next 10 years. In our model, the IPD incidence was reduced from 11.9/100,000 in 2019 to 6.3/100,000 in 2020, caused by a reduced transmission rate due to the COVID-19 mitigation measures. Assuming a recovery in the transmission rate in 2022 April, the incidence of IPD was estimated to increase with maximal incidence of 12.1 and 13.1/100,000 children under 5 years in the rapid and the delayed vaccination recovery scenarios. The difference in the total IPD incidence between these two scenarios was primarily driven by vaccine serotypes IPD incidence. The difference of incidence was not observed between the two scenarios after 2025. The persistent decline in childhood pneumococcal vaccination rates due to the impact of COVID-19 might lead to an increased IPD incidence and an incremental disease burden.