Chest X-ray Pneumonia Detection M.Tech Project: Single-Stage Hybrid CNN-ViT with Cross-Attention Fusion and Lung Masking for TB

About the chest X-ray pneumonia detection M.Tech project

A chest X-ray is the first test for most lung infections, and India carries a heavy load of both pneumonia and tuberculosis. This chest X-ray pneumonia detection M.Tech project reads a chest radiograph and labels it as normal, pneumonia or tuberculosis in one pass, with an attention map drawn only over the lung fields.

The research question: can a CNN and a Vision Transformer that query each other beat both single models and a two-step hybrid, even on X-rays from an unseen hospital?

Research behind it

Base paper. The work starts from From Binary to Multi-Class Classification: A Two-Step Hybrid CNN-ViT Model for Chest Disease Classification Based on X-Ray Images, published in Diagnostics (MDPI, 2024). It fuses a residual CNN with a ViT, first detects TB and then separates TB, viral pneumonia, bacterial pneumonia and normal, using public Kaggle collections.

Research gap. The two-step design needs two passes, and in its multi-class step a plain transformer did slightly better than the hybrid, so the fusion itself can improve. Public collections mix images from different sources, and a model may learn borders, text labels or scanner style rather than lungs. The study also does not test on X-rays from another hospital.

Proposed system and novelty. A U-Net first masks everything outside the lungs. ResNet50 then gives local feature tokens and ViT-Small gives global patch tokens. In a cross-attention block each branch asks the other which regions matter, and the two summaries feed one three-class head. The model is tested without retraining on the Shenzhen and Montgomery TB sets, and Grad-CAM and attention rollout maps are scored by how much of each map lies inside the lung mask.

Models compared. ResNet50, DenseNet121, ViT-Small and the base paper two-step hybrid re-implemented, against the proposed model, on one patient-wise split. The ablation removes lung masking, swaps cross-attention for plain concatenation and keeps one branch at a time.

A similar idea of joining two kinds of features appears in the deepfake image detection project, where spatial and frequency clues are combined.

How it works

• A technician registers the patient and uploads the chest X-ray.
• U-Net finds the lung fields, and the rest of the image is masked.
• The image is enhanced with CLAHE and resized for both branches.
• The hybrid returns normal, pneumonia or TB with a confidence score.
• Disease cases above the limit move to the top of the radiologist's queue.
• Grad-CAM and attention maps are laid over the X-ray.
• The radiologist confirms or corrects the label and downloads a PDF report.

Users and modules

There are three roles. The technician uploads images and patient details and is told when the report is ready. The radiologist works through a priority queue, checks the lung mask and attention map, corrects the label if needed and signs the report. The admin manages users and scans, watches the dashboard, opens the model comparison page and edits settings.

What you get

• Phase 1 Research: Base paper study: two-step hybrid CNN-ViT (Diagnostics, MDPI, 2024); Literature survey of 15-20 recent chest X-ray papers; Research gap, novelty and objectives; TB and pneumonia dataset EDA notebook; Synopsis and Phase-1 report.
• Phase 2 Implementation: Flask web app with radiologist, technician and admin panels; U-Net lung masking and CLAHE pipeline; Baselines and two-step hybrid re-implementation; Proposed cross-attention CNN-ViT model; Grad-CAM and attention rollout module; MySQL database and trained models.
• Results: Accuracy, macro F1, TB recall and ROC-AUC; External test on Shenzhen and Montgomery sets; Ablation graphs and map-in-lung scores; results.xlsx and all graphs in high resolution.
• Thesis and paper: Thesis written for you (about 90 pages); IEEE conference paper draft; Journal paper draft; Publication guide.
• Reviews, viva and support: PPTs for Review 1, Review 2 and final viva; Viva book with 50+ questions; A1 poster and project diary; Demo and code walkthrough videos; 1-on-1 setup, 2 revisions and support till viva.

Who can use this project

This suits M.Tech and M.E. students of CSE and IT who want a medical imaging dissertation with a modern architecture and a clear base paper. Students searching for a hybrid CNN vision transformer project, a cross-attention fusion medical imaging project or a lung segmentation U-Net project will find each part here. Training code is in commented PyTorch notebooks, and the web app loads the saved model on a CPU.