Text classification in low-resource regime is a challenging task. Data augmentation techniques can significantly alleviate the issue of insufficient training samples in such environments by generating new samples. However, existing data augmentation methods have not yet effectively solved the problems of hard samples that are hard to classify and insufficient model generalization ability, which makes the performance of text classification in low-resource regime still has room for improvement. To this end, this paper proposes a method that fuses representational data augmentation and dual supervised contrast learning (RADSCL) in low-resource regime. Representational data augmentation first uses dynamic span-cutoff to remove context-independent words, which reduces the parameters needed for mixup and lowers the computational cost. Then the meaningless property of PAD word embeddings is utilized to perform weighted mixing with the cut-off text to reduce the importance of certain words in the text that affect classification. Through the representation augmentation method proposed in this paper, high-quality hard positive samples can be generated to optimize the decision boundary of the model. On this basis, in view of the current contrastive learning has achieved significant results in enhancing text representation, this paper constructs a dual supervised contrastive learning framework. The framework not only uses supervised contrastive learning to focus on the contrastive relationship between positive and negative samples, but also fully learns the potential semantic relationship between different categories of samples under the role of soft labels. Label distribution contrastive learning further utilizes the distribution information of soft labels based on supervised contrastive learning to impose effective constraints on the text representation, which effectively improves the performance of the text classification model. Multiple sets of experimental results show that the performance of the RADSCL model outperforms any model that incorporates other data augmentation and supervised contrastive learning models by a considerable margin on three benchmark datasets. In particular, the average accuracy of RADSCL is improved by 2.76 %, 2.66 %, 0.91 %, and 2.75 %, respectively, relative to the fusion models such as EDA+CE+SCL, Mixup+CE+SCL, ADV+CE+SCL, and AWD+CE+SCL.


扫码关注我们
求助内容:
应助结果提醒方式:

