The scientific community has joined forces to mitigate the scope of the current COVID-19 pandemic . The early identification of the disease, as well as the evaluation of its evolution is a primary task for the timely application of medical protocols . The use of medical images of the chest provides valuable information to specialists . Specifically, chest X-ray images have been the focus of many investigations that apply artificial intelligence techniques for the automatic classification of this disease . The results achieved to date on the subject are promising . However, some results of these investigations contain errors that must be corrected to obtain appropriate models for clinical use . This research discusses some of the problems found in the current scientific literature on the application of artificial intelligence techniques in the automatic classification of COVID-19 . It is evident that in most of the reviewed works an incorrect evaluation protocol is applied, which leads to overestimating the results.