Proceedings of ... International Joint Conference on Neural Networks. International Joint Conference on Neural Networks最新文献

Transformer-based models have achieved strong performance in single-cell transcriptomics, yet scaling them to genome-wide gene sequences remains challenging due to the quadratic cost of self-attention. Recent state space models (SSMs) offer linear-time sequence modeling, but unidirectional designs may introduce ordering biases and overlook biological structure, potentially limiting robustness across datasets. To address these challenges, we propose GeneMamba, a bidirectional state space model tailored for single-cell RNA sequencing. GeneMamba integrates three key components: (1) Efficiency: BiMamba blocks enable bidirectional processing with O(L) complexity, providing substantial speedup over transformer baselines while maintaining linear scaling; (2) Biological integration: a pathway-aware contrastive objective incorporates curated pathway information to guide representation learning; (3) Scalability: large-scale pretraining on approximately 30 million curated cells (from ~50 million raw CELLXGENE profiles) supports transfer across multiple downstream tasks without task-specific architectures. Across benchmarks including multi-batch integration, cell type annotation, and gene rank reconstruction, GeneMamba demonstrates competitive performance relative to strong baselines such as scGPT and Geneformer, with improvements observed in several settings. Ablation studies further suggest that bidirectional modeling and pathway-aware pretraining contribute to these gains. Overall, GeneMamba provides an efficient and biologically informed framework for foundation modeling in single-cell transcriptomics.

The concept of weight pruning has shown success in neural network model compression with marginal loss in classification performance. However, similar concepts have not been well recognized in improving unsupervised learning. To the best of our knowledge, this paper proposes one of the first studies on weight pruning in unsupervised autoencoder models using non-imaging data points. We adapt the weight pruning concept to investigate the dynamic behavior of weights while reconstructing data using an autoencoder and propose a deterministic model perturbation algorithm based on the weight statistics. The model perturbation at periodic intervals resets a percentage of weight values using a binary weight mask. Experiments across eight non-imaging data sets ranging from gene sequence to swarm behavior data show that only a few periodic perturbations of weights improve the data reconstruction accuracy of autoencoders and additionally introduce model compression. All data sets yield a small portion of (<5%) weights that are substantially higher than the mean weight value. These weights are found to be much more informative than a substantial portion (>90%) of the weights with negative values. In general, the perturbation of low or negative weight values at periodic intervals has improved the data reconstruction loss for most data sets when compared to the case without perturbation. The proposed approach may help explain and correct the dynamic behavior of neural network models in a deterministic way for data reconstruction and obtaining a more accurate representation of latent variables using autoencoders.

Deep neural networks have achieved remarkable success in various challenging tasks. However, the black-box nature of such networks is not acceptable to critical applications, such as healthcare. In particular, the existence of adversarial examples and their overgeneralization to irrelevant, out-of-distribution inputs with high confidence makes it difficult, if not impossible, to explain decisions by such networks. In this paper, we analyze the underlying mechanism of generalization of deep neural networks and propose an (n, k) consensus algorithm which is insensitive to adversarial examples and can reliably reject out-of-distribution samples. Furthermore, the consensus algorithm is able to improve classification accuracy by using multiple trained deep neural networks. To handle the complexity of deep neural networks, we cluster linear approximations of individual models and identify highly correlated clusters among different models to capture feature importance robustly, resulting in improved interpretability. Motivated by the importance of building accurate and interpretable prediction models for healthcare, our experimental results on an ICU dataset show the effectiveness of our algorithm in enhancing both the prediction accuracy and the interpretability of deep neural network models on one-year patient mortality prediction. In particular, while the proposed method maintains similar interpretability as conventional shallow models such as logistic regression, it improves the prediction accuracy significantly.

Semantic features are common radiological traits used to characterize a lesion by a trained radiologist. These features have been recently formulated, quantified on a point scale in the context of lung nodules by our group. Certain radiological semantic traits have been shown to extremely predictive of malignancy [26]. Semantic traits observed by a radiologist at examination describe the nodules and the morphology of the lung nodule shape, size, border, attachment to vessel or pleural wall, location and texture etc. Deep features are numeric descriptors often obtained from a convolutional neural network (CNN) which are widely used for classification and recognition. Deep features may contain information about texture and shape, primarily. Lately, with the advancement of deep learning, convolutional neural networks (CNN) are also being used to analyze lung nodules. In this study, we relate deep features to semantic features by looking for similarity in ability to classify. Deep features were obtained using a transfer learning approach from both an ImageNet pre-trained CNN and our trained CNN architecture. We found that some of the semantic features can be represented by one or more deep features. In this process, we can infer that some deep feature(s) have similar discriminatory ability as semantic features.

Lung cancer is the leading cause of cancer-related deaths globally, which makes early detection and diagnosis a high priority. Computed tomography (CT) is the method of choice for early detection and diagnosis of lung cancer. Radiomics features extracted from CT-detected lung nodules provide a good platform for early detection, diagnosis, and prognosis. In particular when using low dose CT for lung cancer screening, effective use of radiomics can yield a precise non-invasive approach to nodule tracking. Lately, with the advancement of deep learning, convolutional neural networks (CNN) are also being used to analyze lung nodules. In this study, our own trained CNNs, a pre-trained CNN and radiomics features were used for predictive analysis. Using subsets of participants from the National Lung Screening Trial, we investigated if the prediction of nodule malignancy could be further enhanced by an ensemble of classifiers using different feature sets and learning approaches. We extracted probability predictions from our different models on an unseen test set and combined them to generate better predictions. Ensembles were able to yield increased accuracy and area under the receiver operating characteristic curve (AUC). The best-known AUC of 0.96 and accuracy of 89.45% were obtained, which are significant improvements over the previous best AUC of 0.87 and accuracy of 76.79%.

Magnetic Resonance Imaging (MRI) is one of the leading modalities for medical imaging, providing excellent soft-tissue contrast without exposure to ionizing radiation. Despite continuing advances in MRI, long scan times remain a major limitation in clinical applications. Parallel imaging is a technique for scan time acceleration in MRI, which utilizes the spatial variations in the reception profiles of receiver coil arrays to reconstruct images from undersampled Fourier space, i.e. k-space. One of the most commonly used parallel imaging techniques employs interpolation of missing k-space information by using linear shift-invariant convolutional kernels. These kernels are trained on a limited amount of autocalibration signal (ACS) for each scan. We propose a novel method for parallel imaging, Robust Artificial-neural-networks for k-space Interpolation (RAKI), which uses scan-specific convolutional neural networks (CNNs) to perform improved k-space interpolation. Three-layer CNNs are trained using only scan-specific ACS data, alleviating the need for large training databases. The proposed method was tested in ultra-high resolution brain MRI and quantitative cardiac MRI, acquired with various acceleration rates. Improved noise resilience as compared to existing parallel imaging methods was observed for high acceleration rates or in the presence of low signal-to-noise ratio (SNR). Furthermore, RAKI successfully reconstructed images for quantitative cardiac MRI, even when using the same CNN across images with varying contrasts. These results indicate that RAKI achieves improved noise performance without overfitting to specific image contents, and offers great promise for improved acceleration in a wide range of MRI applications.

Recent advances in medical imaging technologies generate a high volume of imaging data. Classification of cognitive outcome and disease status based on brain images is one of the most important tasks in neuroimaging studies. However it poses great challenge to the current classification methods due to the extremely high dimensionality and low signal to noise ratio in brain image data. In this article we propose a tensor boosting algorithm for classification based on neuroimaging data. The method is off-the-shelf, computationally simple and amenable to various modalities of neuroimaging data. The method is applied to an EEG data set from an alcoholism study and an MRI data set from an ADHD Global Competition and shows significantly improved classification performance.

Computational protein structure prediction is very important for many applications in bioinformatics. In the process of predicting protein structures, it is essential to accurately assess the quality of generated models. Although many single-model quality assessment (QA) methods have been developed, their accuracy is not high enough for most real applications. In this paper, a new approach based on C-α atoms distance matrix and machine learning methods is proposed for single-model QA and the identification of native-like models. Different from existing energy/scoring functions and consensus approaches, this new approach is purely geometry based. Furthermore, a novel algorithm based on deep learning techniques, called DL-Pro, is proposed. For a protein model, DL-Pro uses its distance matrix that contains pairwise distances between two residues' C-α atoms in the model, which sometimes is also called contact map, as an orientation-independent representation. From training examples of distance matrices corresponding to good and bad models, DL-Pro learns a stacked autoencoder network as a classifier. In experiments on selected targets from the Critical Assessment of Structure Prediction (CASP) competition, DL-Pro obtained promising results, outperforming state-of-the-art energy/scoring functions, including OPUS-CA, DOPE, DFIRE, and RW.

Though treatment of the ventilated premature infant has experienced many advances over the past decades, determining the best time point for extubation of these infants remains challenging and the incidence of extubation failures largely unchanged. The objective was to provide clinicians with a decision-support tool to determine whether to extubate a mechanically ventilated premature infant by using a set of machine learning algorithms on a dataset assembled from 486 premature infants receiving mechanical ventilation. Algorithms included artificial neural networks (ANN), support vector machine (SVM), naïve Bayesian classifier (NBC), boosted decision trees (BDT), and multivariable logistic regression (MLR). Results for ANN, MLR, and NBC were satisfactory (area under the curve [AUC]: 0.63-0.76); however, SVM and BDT consistently showed poor performance (AUC ~0.5). Complex medical data such as the data set used for this study require further preprocessing steps before prediction models can be developed that achieve similar or better performance than clinicians.

This paper introduces a prototype framework for content-based EEG retrieval (CBER). Like content-based image retrieval, the proposed framework retrieves EEG segments similar to the query EEG segment in a large database. Such retrieval of EEG can be used to assist data mining of brain signals by allowing researchers to understand the association between brain patterns, responses, and the environment. Retrieval might also be used to enhance the accuracy of Brain Computer Interface (BCI) systems by providing related samples for training. We present key components of CBER and explain how to handle the distinctive characteristics of EEG. To demonstrate the feasibility of the framework, we implemented a simple EEG database of about 37,000 samples from more than 100 subjects. We ran two retrieval scenarios with a set of EEG features and evaluation metrics. The results of finding similar subjects clearly demonstrate the potential of CBER in many EEG applications.