Sarcasm Detection: A Comparative Analysis of RoBERTa-CNN vs RoBERTa-RNN Architectures

Sheraton Pawestri, Murinto Murinto, Muhammad Auzan

Abstract

Increasingly advanced technology and the creation of social media and the internet can become a forum for people to express things or opinions. However, comments or views from users sometimes contain sarcasm making it more difficult to understand. News headlines, sometimes contain sarcasm which makes readers confused about the content of the news. Therefore, in this research, a model was created for sarcasm detection. Many methods are used for sarcasm detection, but performance still needs to be improved. So this research aims to compare the performance of two text classification methods, Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN), in detecting sarcasm in English news headlines using RoBERTa text transformation.  RoBERTa produces a fixed-size vector of numbers 1x768. The research results show that CNN has better performance than RNN. CNN achieved the highest average accuracy of 0.891, precision of 0.878, recall of 0.874, and f1-score of 0.876, with a loss of 0.260 and a processing time of 508.1 milliseconds per epoch. On the contrary, RNN shows an accuracy of 0.711, precision of 0.692, recall of 0.620, f1-score 0.654, and loss of 0.564, with a longer processing time of 116500 milliseconds per epoch. The 10-fold cross-validation evaluation method ensures the model performs well and avoids overfitting. So it is recommended to use the combination of RoBERTa and CNN in other text classification applications that require high speed and accuracy. Further research is recommended to explore deeper CNN architectures or other architectural variations such as Transformer-based models for performance improvements.

Full Text:

PDF (118-125)

References

M. Nur Fitriyah and Y. Effendri, “BAHASA SARKASME WARGANET DALAM KOMENTAR AKUN INSTAGRAM PUAN MAHARANI DAN DPR RI,” Bapala, vol. 8, no. 4, pp. 112–119, 2021.

A. Azis and L. Marlina, “Analysis of Sarcasm Found in Keith Alberstadt’s Comedy,” English Language and Literature, vol. 9, no. 2, p. 215, Aug. 2020, doi: 10.24036/ell.v9i2.7805.

R. J. Kreuz and S. Glucksberg, “How to be sarcastic: The echoic reminder theory of verbal irony.,” J Exp Psychol Gen, vol. 118, no. 4, pp. 374–386, Dec. 1989, doi: 10.1037/0096-3445.118.4.374.

P. V. Dauphin, “SARCASM IN RELATIONSHIPS.” Accessed: Oct. 07, 2023. [Online]. Available: https://ccat.sas.upenn.edu/plc/communication/valerie.htm

F. Anshari and A. Hafiz, “Bahasa Sarkasme dalam Berita Olahraga - Studi Kasus bolatory.com,” Prosiding Konferensi Nasional Komunikasi, vol. 2, no. 1, pp. 184–196, 2018.

S. Masroor and M. S. Husain, “Sarcasm Analysis Using Social Media : A Literature Review,” IJEMR, pp. 26–31, 2018.

F. Ardiansyah and A. Kartikadewi, “Sarcasm Detection on Indonesian Politics Tweet Using Multi Labeling Method and Support Vector Machine,” International Journal of Engineering and Techniques, vol. 6, no. 3, Jun. 2020, doi: 10.29126/23951303/IJET-V6I3P15.

C. S. Arsya and M. Elsera, “IMPLEMENTASI RANDOM FOREST DALAM MELAKUKAN KLASIFIKASI KATA SARKASME PADA MEDIA SOSIAL FACEBOOK,” Djtechno: Jurnal Teknologi Informasi, vol. 4, no. 1, pp. 216–223, Jul. 2023, doi: 10.46576/djtechno.v4i1.3361.

D. Alita, S. Priyanta, and N. Rokhman, “Analysis of Emoticon and Sarcasm Effect on Sentiment Analysis of Indonesian Language on Twitter,” Journal of Information Systems Engineering and Business Intelligence, vol. 5, no. 2, p. 100, Oct. 2019, doi: 10.20473/jisebi.5.2.100-109.

R. A. Bagate and R. Suguna, “Different Approaches in Sarcasm Detection: A Survey,” 2020, pp. 425–433. doi: 10.1007/978-3-030-34080-3_48.

D. Hazarika, S. Poria, S. Gorantla, E. Cambria, R. Zimmermann, and R. Mihalcea, “CASCADE: Contextual Sarcasm Detection in Online Discussion Forums,” in Proceedings of the 27th International Conference on Computational Linguistics, E. M. Bender, L. Derczynski, and P. Isabelle, Eds., Santa Fe, New Mexico, USA: Association for Computational Linguistics, Aug. 2018, pp. 1837–1848. [Online]. Available: https://aclanthology.org/C18-1156

R. Misra and P. Arora, “Sarcasm Detection using Hybrid Neural Network,” Aug. 2019, doi: 10.13140/RG.2.2.32427.39204.

R. Misra and P. Arora, “Sarcasm detection using news headlines dataset,” AI Open, vol. 4, pp. 13–18, 2023, doi: 10.1016/j.aiopen.2023.01.001.

A. C. Wijaya and I. G. A. Wibawa, “ Deteksi Sarkasme dan Ironi pada Twitter dengan Mengunakan Metode CNN,” Jurnal Nasional Teknologi Informasi dan Aplikasnya, vol. 1, no. 1, pp. 353–360, 2022, [Online]. Available: https://ojs.unud.ac.id/index.php/jnatia/article/view/92591

N. Jaiswal, “Neural Sarcasm Detection using Conversation Context,” in Proceedings of the Second Workshop on Figurative Language Processing, Stroudsburg, PA, USA: Association for Computational Linguistics, 2020, pp. 77–82. doi: 10.18653/v1/2020.figlang-1.11.

H. Yaghoobian, H. R. Arabnia, and K. Rasheed, “Sarcasm Detection: A Comparative Study,” Jul. 2021.

M. D. Hilmawan, “Deteksi Sarkasme Pada Judul Berita Berbahasa Inggris Menggunakan Algoritme Bidirectional LSTM,” Journal of Dinda : Data Science, Information Technology, and Data Analytics, vol. 2, no. 1, pp. 46–51, Feb. 2022, doi: 10.20895/dinda.v2i1.331.

Y. Yunitasari, A. Musdholifah, and A. K. Sari, “Sarcasm Detection For Sentiment Analysis in Indonesian Tweets,” IJCCS (Indonesian Journal of Computing and Cybernetics Systems), vol. 13, no. 1, p. 53, Jan. 2019, doi: 10.22146/ijccs.41136.

O. Zalutska et al., “Method for Sentiment Analysis of Ukrainian-Language Reviews in E-Commerce Using RoBERTa Neural Network,” in International Conference on Computational Linguistics and Intelligent Systems, 2023. [Online]. Available: https://api.semanticscholar.org/CorpusID:258688336

K. O’Shea and R. Nash, “An Introduction to Convolutional Neural Networks,” Nov. 2015.

A. Sherstinsky, “Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) network,” Physica D, vol. 404, p. 132306, Mar. 2020, doi: 10.1016/j.physd.2019.132306.

Y. Qiao, C. Xiong, Z. Liu, and Z. Liu, “Understanding the Behaviors of BERT in Ranking,” Apr. 2019.

S. F. Chaerul Haviana, S. Mulyono, and Badie’Ah, “The Effects of Stopwords, Stemming, and Lemmatization on Pre-trained Language Models for Text Classification: A Technical Study,” in 2023 10th International Conference on Electrical Engineering, Computer Science and Informatics (EECSI), 2023, pp. 521–527. doi: 10.1109/EECSI59885.2023.10295797.

K. Fukushima, “Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position,” Biol Cybern, vol. 36, no. 4, pp. 193–202, 1980, doi: 10.1007/BF00344251.

Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE, vol. 86, no. 11, pp. 2278–2324, 1998, doi: 10.1109/5.726791.

J. J. Hopfield, “Neural networks and physical systems with emergent collective computational abilities.,” Proceedings of the National Academy of Sciences, vol. 79, no. 8, pp. 2554–2558, Apr. 1982, doi: 10.1073/pnas.79.8.2554.

D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning representations by back-propagating errors,” Nature, vol. 323, no. 6088, pp. 533–536, 1986, doi: 10.1038/323533a0.

S. Hochreiter and J. Schmidhuber, “Long Short-Term Memory,” Neural Comput, vol. 9, no. 8, pp. 1735–1780, Nov. 1997, doi: 10.1162/neco.1997.9.8.1735.

Y. Liu et al., “RoBERTa: A Robustly Optimized BERT Pretraining Approach,” Jul. 2019.

DOI: https://doi.org/10.37058/innovatics.v6i2.11921

Refbacks

  • There are currently no refbacks.