In this project, a transformer based Deep Learning model has been developed that could predict the Enzyme Commission Numbers of Enzymes from full scale sequences. The input of the model is full-scale enzyme protein sequences, and the output are the four EC numbers.
Left: Data Flow of the training loop for the ProtEC model; Right: A Block diagram of the overall architecture of the ProtEC model.
Model:
The model consists of four modified ProtBert modules which has been slectively finetuned to achieve state of the art accuracy on EC numbers. The model has been trained on the Uniprot Swissprot reviewed dataset on two types of splits: The first one is called the random split, which randomly divides the data into train, validation, and testing sets. Whereas, the second type is called clustered split, where the data has been split using UNIREF to make sure that the training and testing splits consists of different distributions of sequences.
Results:
The model has been compared with Proteinfer by Goodle and performs better in terms of accuracy and F1 scores. Moreover, the model is able to retain very high accuracy even when the training dataset is shrunk to 10% of its original size, making it suitable in applications with a very low amount of data. Fruthermore, the model accuracy is independant of sequence lengt,h so it is able to preform with very long or short sequences. Lastly, the model tunes most of its hyperparameters by itself, so it is easy to use and does not require a separate validation set in order to train.
The advancements in next-generation sequencing technologies have given rise to large-scale, open-source protein databases consisting of hundreds of millions of sequences. However, to make these sequences useful in biomedical applications, they need to be painstakingly annotated by curating them from literature. To counteract this problem, many automated annotation algorithms have been developed over the years including deep learning models, especially in recent times. In this work, we propose a transformer-based deep-learning model that can predict the Enzyme Commission numbers of an enzyme from full-scale sequences with state-of-the-art accuracy compared to other recent machine learning annotation algorithms. The system does especially well on clustered split dataset which consists of training and testing samples derived from different distributions that are structurally dissimilar from each other. This proves that the model is able to understand deep patterns within the sequences and can accurately identify the motifs responsible for the different enzyme commission numbers. Moreover, the algorithm is able to retain similar accuracy even when the training size is significantly reduced, and also, the model accuracy is independent of the sequence length making it suitable for a wide range of applications consisting of varying sequence structures.
@article{ProtEC,author={Tamir, Azwad and Salem, Milad and Yuan, Jiann-Shiun},journal={IEEE/ACM Transactions on Computational Biology and Bioinformatics},title={ProtEC: A Transformer Based Deep Learning System for Accurate Annotation of Enzyme Commission Numbers},year={2023},volume={20},number={6},pages={3691-3702},keywords={Proteins;Computational modeling;Transformers;Databases;Annotations;Enzymes;Biological system modeling;Deep learning;enzyme commission numbers;machine learning;ProtBert;protein annotation;protein sequences;transformers},doi={10.1109/TCBB.2023.3311427},}
ProtEC: A Transformer Based Deep Learning System for Accurate Annotation of Enzyme Commission NumbersIEEE/ACM Transactions on Computational Biology and Bioinformatics. Publication link , Mar 2023