Abstract:

Abstract With the immense growth in the field of computational algorithms and data management, the demand for automating the medical analysis and diagnosis is also increasing. The foundational demand from the medical analysis is rapid analysis with least error or almost with zero errors. The manual process is subjected to the higher human interventions and with higher scope of errors. Henceforth, dealing with analysis of life treating diseases such as lung cancer must be automated. The challenge with the computer driven automated processes is the quality of the data decides the accuracy of the final outcomes or information. Henceforth, the data cleaning or as called literally data pre-processing is one of the major focused areas of concern for building automated frameworks for disease detections. Many Researchers have dedicatedly worked towards achieving the best pre-processing framework. Nonetheless, these research attempts are criticised for various reasons such not designed for medical information pre-processing as various parameters like precision, “missing value” and dimension of the data plays a major role. Few of parallel research outcomes have demonstrated higher focus on the medical information pre-processing while building the framework. However, these methods demonstrate higher complexity and hard to adapt due to strong dependency on the “dataset”. Henceforth, the paper proposes a novel framework for medical data pre-processing with few benchmarking proposed algorithms with adaptive and threshold driven method for “outlier” detection and imputation, domain specific “missing value” detection and imputation, and finally mete information specific noise reduction. The outcome of the proposed framework demonstrates nearly 50% improvement with the benchmarked algorithms attached with the proposed framework due to this adaptation.     

Keywords: