2.1 Dataset

This study made use of a gene expression dataset obtained from TCGA website, which provides access to a wide range of biomedical datasets, including mRNA data. The dataset, which included information from 1,157 kidney cancer patients, was meticulously prepared for analysis. Although the original dataset included both gene expression and clinical data, only gene expression data was used in this study. To ensure accuracy and consistency, the dataset was cleaned and preprocessed to remove missing, duplicate, and invalid values, as well as clinical data. tables 1 and 2 provide detailed statistics about the dataset, such as the total number of datasets for each stage and information on data features. The "Before Cleaning" row refers to the original dataset, which was downloaded from the TCGA website without any preprocessing, whereas the "After Cleaning" row refers to the dataset after various cleaning and preprocessing techniques were applied. This meticulous preparation of the gene expression dataset contributed to the subsequent analyses producing reliable and meaningful results.

2.2 Correlation test

Naturally, a gene expression dataset consists of a large number of features, and each feature represents a unique gene of a patient. Among this massive number of features, some are highly correlated, while others are less correlated or not at all. By doing a correlation test on the dataset, we can significantly improve the performance of the classification models ⁽¹⁵⁾. Classification models might be hampered by redundant features. As a result, the Pearson correlation coefficient-based method is utilized to identify redundant features, which leads to their removal. We choose only the top 1000 features with the least amount of redundancy after filtering the data using Pearson correlation values. The following formula defines the Pearson correlation values:

$r=\frac{\sum\left(x_i-\bar{x}\right)\left(y_i-\bar{y}\right)}{\sqrt{\sum\left(x_i-\bar{x}\right)^2 \sum\left(y_i-\bar{y}\right)^2}}$

2.3 Feature selection

After features were filtered using the correlation test, a second method was used to reduce the dimension of the features by applying the feature selection method. Feature selection can help in getting rid of irrelevant data as well as dealing with noise in the dataset ⁽¹⁶⁾, ⁽¹⁷⁾. Two feature selection techniques were selected and combined to choose a list of highly relevant features from thousands of features. These techniques are Least Absolute Shrinkage and Selection Operator (LASSO) and Analysis of Variance (ANOVA). Lasso is a linear regression in which shrinkage techniques are used to shrink the coefficients of determination toward zero ⁽¹⁸⁾. ANOVA is a famous statistical hypothesis test used to determine whether there are any significant differences in the variances of two or more groups ⁽¹⁹⁾.

Three different experiments, including using only LASSO, using only ANOVA, and combining both LASSO and ANOVA together, were conducted to find the best feature selection strategy that gave the best classification result. By evaluating the results produced by these three strategies, we can observe that combining LASSO with ANOVA can improve classification performance. We combined these two techniques' results by selecting the first 500 relevant features from the intersection of both results.

2.4 Feature extraction

Two popular methods can be used for dimension reduction when the dataset contains many features. Besides the earlier mentioned feature selection technique, feature extraction is another way to do it. The main goal of feature extraction is to create a smaller version of the original dataset while keeping the meaning of the original dataset ⁽²⁰⁾. Besides dimension reduction, improving training speed and better computation are well-known feature extraction advantages ⁽²¹⁾–⁽²³⁾. This research used two feature extraction techniques: AE, VAE.

2.5 Synthetic minority over-sampling technique

Our dataset is skewed or imbalanced, which means that the number of observations for each class is not equal or close to each other. table 1 and 2, which describe the data statistics, show that the observations for tumor stage 1 cover around 50% of the dataset, leaving another 50% for the other three tumor stages. Imbalanced data can cause problems for machine learning models by biasing the model toward any class with more datasets ⁽²⁸⁾–⁽³⁰⁾. To solve this problem, we apply an oversampling technique called Synthetic Minority Over-sampling Technique (SMOTE) to generate a new set of datasets that contain the same amount of observation for all classes ⁽³¹⁾. Prior to utilizing SMOTE, the train dataset exhibited an unequal distribution across the kidney tumor stages. Stage 1 had the highest count with 312 records, followed by Stage 3 with 189, Stage 2 with 105, and Stage 4 being the least represented with 76 instances. In a bid to rectify this imbalance, we applied the SMOTE technique with the "auto" hyperparameter, which adopts an adaptive sampling strategy. After the SMOTE application, the total count of records for all tumor stages combined equaled 1248, with each individual stage-Stage 1 to Stage 4-having a balanced representation of 312 records.

2.6 Classifiers

We applied the top 9 classification algorithms, allowing us to assess how well they performed against one another. These methods include logistic regression (LR) ⁽³²⁾, support vector machine (SVM) ⁽³³⁾, decision tree (DT) ⁽³⁴⁾, random forest (RF) ⁽³⁵⁾, k-nearest neighbor (KNN) ⁽³⁶⁾, naïve bayes (NB) ⁽³⁷⁾, AdaBoost (ADA) ⁽³⁸⁾, XGBoost (XGB) ⁽³⁹⁾, and stochastic gradient descent classifier (SGD) ⁽⁴⁰⁾. For the selected classifiers, the hyperparameter configurations are as follows: For LR, the settings are C=1, penalty="l2", and solver="liblinear". SVM is set with C=1, kernel="rbf", and gamma="scale". DT is configured with max_depth=5. RF uses n_estimators=100 and max_depth=5. KNN utilizes n_neighbors=5. NB operates with default settings. ADA uses n_estimators=50. XGB is set with n_estimators=100 and learning_rate=0.1. For SGD, it's wrapped in a calibration with max_iter=1000 and tol=1e-3.

2.7 Model evaluation metrics

This section discusses the evaluation metrics for all the classifiers listed above. The seven most popular evaluation metrics were used to compare the performance of those classifiers, including accuracy, recall, precision, the f1-score, sensitivity, specificity, and area under the curve (AUC) ⁽⁴¹⁾. To balance the model performance across classes, we use macro-averaged precision, recall, and f1-score to get the overall average value for each metric. Besides these metrics, specificity, and sensitivity were also calculated to get a deeper understanding of the rates of true positive and true negative, respectively. In the below formula: TP, TN, FP, and FN are the numbers of true positive, true negative, false positive, and false negative, respectively.

$\begin{aligned} & \text { Accuracy }=\frac{T P+T N}{T P+T N+F P+F N} \\ & \text { Precision }=\frac{T P}{T P+F P} \\ & \text { Recall }=\frac{T P}{T P+F N} \\ & \text { F1-score }=\frac{2 \times \text { Recall } \times \text { Precision }}{\text { Recall }+ \text { Precision }} \\ & \text { Sensitivity }=\frac{T P}{T P+F N} \\ & \text { Specificity }=\frac{T N}{T N+F P}\end{aligned}$

3.1 Feature extraction with deep learning

We reduced the total number of features from thousands to 1000 by filtering using a correlation test. Then, we continue reducing the dataset dimension from 1000 to 500 by applying feature selection techniques. From these 500 features, we applied features extraction to reduce the dataset dimension to 50 features. We chose three feature extraction algorithms to conduct the experiment, including AE and VAE. Based on the experiment result, the original and reconstructed dataset was observed to be similar, which shows the efficiency in feature extraction. fig 4shows the models' loss values, and fig 5visualizes the reconstructed data point compared to the original data point when using AE and VAE models. Among these two algorithms, the AE model produced the most similar reconstructed dataset compared to the original dataset.

3.2 Classification results

This section shows the result of 9 machine learning classifiers that is used for classifying patient kidney tumor stages. Before training classifiers, we preprocessed data with correlation analysis, feature selection, feature extraction, and oversampling for the purpose of improving classification performance. table 3 and 4 indicate the evaluation metrics of all classifiers when using an AE and VAE as feature extraction with and without the oversampling technique applied. In these two tables, the "Sampling" column indicates whether the oversampling technique was applied.

table 3 shows the performance of all classifiers when using the AE model to extract the features. In this case, the support vector machine outperformed other classifiers in most evaluation metrics. XGBoost, Naïve Bayes, and Decision Tree are the next algorithms to show good results. We could see that SVM significantly improves the performance after applying the oversampling technique to the dataset.

Moreover, another different result is shown in table 4, which indicates the result when we used a VAE to extract the features. The performance changed when we changed the feature extraction algorithm, which means it decreased in this case compared to the AE model. The table shows that SVM still outperforms most of the classifiers in terms of AUC value, followed by XGBoost, Naïve Bayes, and Decision Tree.

tables 3 and 4 show the evaluation result of all classifiers as an average of four tumor classes. fig 6can bring more detail to the Area Under the Curve (AUC) value listed in the above tables by showing the Receiver operating characteristic curve (ROC) and AUC value for each tumor stage without calculating it as an average value. We only show the ROC curve when AE is used as feature extraction since we already observed that AE is outperformed the other two feature extraction methods.