Here we’ll compare the results we’ve obtained with those that had been published for alternative models, as well as analyze its performance on a new dataset.
In this series of articles, we’ll apply a Deep Learning (DL) network, ResNet50, to diagnose Covid-19 in chest X-ray images. We’ll use Python’s TensorFlow library to train the neural network on a Jupyter Notebook.
The tools and libraries you’ll need for this project are:
IDE:
Libraries:
We are assuming that you are familiar with deep learning with Python and Jupyter notebooks. If you're new to Python, start with this tutorial. And if you aren't yet familiar with Jupyter, start here.
In the previous article, we trained and tested a ResNet50 model that had been transfer-learned to classify chest X-rays into COVID-19 and Normal images. In this article, we’ll compare the results we’ve obtained with those that had been published for alternative models, as well as analyze its performance on a new dataset.
Result Comparison
As was mentioned in the previous article, our network showed a robust performance when classifying testing images, in which it achieved the accuracy of 95%.
Table 1 shows a comparison of our network’s accuracy with other related competing solutions. You can see that our fine-tuned ResNet50 outperformed several networks in diagnosing COVID-19. This demonstrates the powerful generalization capability of our model.
| Reference | Pre-trained Model | Dataset | Testing Accuracy |
| Panwar (2020) | VGG16 | COVID-19 and others | 88.1% |
| Albahli (2020) | ResNet152 | COVID-19 and other chest diseases | 87% |
| Ozturk (2020) | DarkCovidNet | COVID-19, Pneumonia, and Normal | 87% |
| Ours | ResNet50 | COVID-19 and Normal | 95% |
Testing on a New Dataset
To further verify the feasibility of the proposed transfer learning-based COVID-19 diagnosis system, we tested it on a new set of images, collected from another public dataset. Testing a network on a totally new dataset can be challenging as the type and quality of images may be different than those of the first dataset that the network was trained on. We usually test the network performance on a small set of images taken from the same dataset used for training. However, in this project, we attempted to measure the robustness of the network if tested on new images taken from a new dataset. The new dataset contains COVID-19 and No finding images but we only selected 300 COVID-19 images and passed them to our model. First, we load the test images from the new dataset using ImageDataGenerator.
test_generator2 = train_datagen.flow_from_directory(r'C:\Users\abdul\Desktop\ContentLab\test2',
target_size = (224, 224),
color_mode = 'rgb',
batch_size = 3,
class_mode = 'Binary',
shuffle = True)
After loading the new testing images, we passed them to the model to calculate the accuracy:
Testresults2 = model.evaluate(test_generator2)
print("test2 loss, test2 acc:", Testresults2)
It is clear that our model maintained a relatively good accuracy even when it was run on images from a new dataset.
Next Step
In the next article, we’ll show you how to build a network for Covid-19 detection from scratch. Stay tuned!
