ProtGO | Azwad Tamir

Repository: GitHub

Paper Link

ProtGO: A Transformer based Fusion Model for accurately predicting Gene Ontology (GO) Terms from full scale Protein Sequences

Paper ref: (Tamir & Yuan, 2024)

Description:

In this project, a transformer based parallal modular fusion model has been developed that could predict the Gene Ontology (GO Terms) of proteins from full-scale sequences. The input of the model is full-scale enzyme protein sequences, and the output is the four EC numbers.

A Block diagram of the overall architecture of the ProtGO model.

Recent developments in next-generation sequencing technology have led to the creation of extensive, open-source protein databases consisting of hundreds of millions of sequences. To render these sequences applicable in biomedical applications, they must be meticulously annotated by wet lab testing or by extracting them from existing literature. Over the last few years, researchers have developed numerous automatic annotation systems, particularly deep learning models based on machine learning and artificial intelligence, to address this issue.

In this work, we propose a transformer-based fusion model capable of predicting Gene Ontology (GO) terms from full-scale protein sequences, achieving state-of-the-art accuracy compared to other contemporary machine learning annotation systems. The approach performs particularly well on clustered split datasets, which comprise training and testing samples originating from distinct distributions that are structurally diverse. This demonstrates that the model can understand both short and long-term dependencies within the enzyme’s structure and can precisely identify the motifs associated with the various GO terms. Furthermore, the technique is lightweight and less computationally expensive compared to the benchmark methods, while at the same time not being affected by sequence length, rendering it appropriate for diverse applications with varying sequence lengths.

Dataset link: https://drive.google.com/file/d/19SXO7Asy2vsAab6cl36-IkUqq9DYMgDb/view?usp=sharing

References

2024

Protein-AI
ProtGO: A Transformer based Fusion Model for accurately predicting Gene Ontology (GO) Terms from full scale Protein Sequences

Azwad Tamir and Jiann-Shiun Yuan

. Publication link , Mar 2024

Abs Bib PDF

Recent developments in next-generation sequencing technology have led to the creation of extensive, open-source protein databases consisting of hundreds of millions of sequences. To render these sequences applicable in biomedical applications, they must be meticulously annotated by wet lab testing or extracting them from existing literature. Over the last few years, researchers have developed numerous automatic annotation systems, particularly deep learning models based on machine learning and artifcial intelligence, to address this issue. In this work, we propose a transformer-based fusion model capable of predicting Gene Ontology (GO) terms from full-scale protein sequences, achieving state-of-the-art accuracy compared to other contemporary machine learning annotation systems. The approach performs particularly well on clustered split datasets, which comprise training and testing samples originating from distinct distributions that are structurally diverse. This demonstrates that the model is able to understand both short and long term dependencies within the enzyme’s structure and can precisely identify the motifs associated with the various GO terms. Furthermore, the technique is lightweight and less computationally expensive compared to the benchmark methods, while at the same time not unaffected by sequence length, rendering it appropriate for diverse applications with varying sequence lengths.
@misc{ProtGO, title = {ProtGO: A Transformer based Fusion Model for accurately predicting Gene Ontology (GO) Terms from full scale Protein Sequences}, author = {Tamir, Azwad and Yuan, Jiann-Shiun}, year = {2024}, eprint = {2412.05776}, archiveprefix = {arXiv}, primaryclass = {cs.LG}, url = {https://arxiv.org/abs/2412.05776}, keywords = {Artifcial Intelligence;Attention;BERT;Deep Learning;fine-tuning;Fusion models;Gene Ontology(GO) terms;Machine Learning;ProtBert;Pretraining;Protein annotation;Protein sequences;Selective Transfer Learning;Transformers}, }