RNA碱基不成对概率比赛的基线模型

赛题介绍

“RNA碱基不成对概率”衡量了RNA序列在各个点位是否能形成稳定的碱基对（base pair），是RNA结构的重要属性，并可被应用在mRNA疫苗序列设计、药物研发等领域。例如mRNA疫苗序列通常不稳定，而RNA碱基不成对概率较高的点位正是易被降解的位置；又如RNA 碱基不成对概率较高的点位通常更容易与其他RNA序列相互作用，形成RNA-RNA binding等，这一特性也被广泛应用于疾病诊断和RNA药物研发。

本次比赛提供了5000条训练数据，请选手基于训练数据和飞桨平台，开发模型预测RNA碱基不成对概率。

（Tips：机器学习框架方面只允许使用飞桨深度学习框架哦）

竞赛数据集

基线系统代码结构

本次基线基于飞桨PaddlePaddle2.0版本。

训练脚本

python src/main.py train --model-path-base [model_directory_name]

本代码会训练一个模型，并且保存到指定位置，训练日志默认保存到文件train_log.txt
注意：由于初始化的不稳定，可能需要多次训练，比较合理的验证集(dev)均方误差损失值(MSE loss)为0.05-0.08

样例

python src/main.py train --model-path-base model

你将会看到类似如下的训练日志

epoch 1 batch 40 processed 640 batch-loss 0.1984 epoch-elapsed 0h00m10s total-elapsed 0h00m11s 
epoch 1 batch 41 processed 656 batch-loss 0.2119 epoch-elapsed 0h00m10s total-elapsed 0h00m11s 
epoch 1 batch 42 processed 672 batch-loss 0.2205 epoch-elapsed 0h00m11s total-elapsed 0h00m11s 
epoch 1 batch 43 processed 688 batch-loss 0.2128 epoch-elapsed 0h00m11s total-elapsed 0h00m11s 
# Dev Average Loss: 0.212 (MSE) -> 0.461 (RMSD)

注意事项

请使用GPU版本的配置环境运行本模块

预测脚本

python src/main.py test --model-path-base [saved_model_directory]

本代码会预测一个模型，日志和结果默认保存到文件test_log.txt

样例

1. 用不带标签的测试集来预测：
   python src/main.py test --model-path-base model-0/model_dev\=0.0772/
2. 用带标签的测试集来预测并评估:
   python src/main.py test_withlabel --model-path-base model-0/model_dev\=0.0772/
   样例输出                

   # python3 src/main.py test_withlabel --model-path-base model-0/model_dev=0.0772Loading data...Loading model...initializing vacabularies... done.Sequence(6): ['<START>', '<STOP>', 'A', 'C', 'G', 'U']Brackets(5): ['<START>', '<STOP>', '(', ')', '.']W0113 21:57:44.871776   221 device_context.cc:252] Please NOTE: device: 0, CUDA Capability: 70, Driver API Version: 11.0, Runtime API Version: 9.0W0113 21:57:44.878015   221 device_context.cc:260] device: 0, cuDNN Version: 7.6.#  Dev Average Loss: 0.0772 (MSE) -> 0.2778 (RMSD)# Test Average Loss: 0.0445 (MSE) -> 0.2111 (RMSD)

   登录后复制
              

• 由于比赛的公开数据不提供测试集的标签，故本基线模型无法运行预设的test_withlabel，除非用户自己生成一个带标签的测试集~/data/test.txt。

注意事项

请使用GPU版本的配置环境运行本模块

写在最后

选手可在基线模型基础上持续改进，也可以采用全新的模型。可以尝试从以下几方面来进行调优:

• 输入数据预处理，提取更多feature
  • 基线模型使用LinearFold预测的RNA二级结构作为辅助feature。选手可以尝试增加更多的辅助feature，如：使用其他二级结构预测软件（如Vienna RNAfold, RNAstructure, CONTRAfold等）生成新的二级结构feature。
• 更复杂的Embedding形式
• 可以尝试在Embedding层使用Elmo, Bert等预训练模型
• 优化网络结构和参数
  • 隐层大小选择 - 宽度和层数
  • 尝试复杂网络构建
  • 尝试正则化、dropout等方式避免过拟合
  • 选择学习率等超参数
  • 选择合适的损失函数
  • 尝试不同的优化器

点击跳转到 中文 说明

Baseline for RNA base unpaired probability

RNA structure prediction contest: The baseline model of RNA base unpairing probability prediction is now released. Using a very simple network model, you can get good results. Developers are welcome to contribute better baseline works~

Calliper 文档对比神器

文档内容对比神器

28

查看详情

About the challange

"RNA base unpaired probability" measures whether the RNA sequence can form a stable base pair at certain point. It is an important attribute of RNA structure and can be used in the design of mRNA vaccine sequence, drug development and other fields. For example, mRNA vaccine sequences are usually unstable, and the points with a higher probability of unpaired RNA bases are the positions that are easily degraded; another example is the points with higher probability of unpaired RNA bases are usually more likely to interact with other RNA sequences, with the formation of RNA-RNA binding, etc. This feature is also widely used in disease diagnosis and RNA drug development.

This competition provides 5000 training data. You are asked to develop a model to predict the unpaired probability of RNA bases, given the training data and PaddlePaddle platform.

（Tips：Machine learning models can be only developed with PaddlePaddle deep learning frameworks）

Dataset for RNA contest

Baseline system

This baseline is developed with PaddlePaddle2.0.

To train

python src/main.py train --model-path-base [model_directory_name]

This will train a paddle model and save it in the specified directory. Training log will be outputed to train_log.txt by default.
Note: You may need to train a few times, as the model might have bad initialization and may not train well. (A dev MSE loss of about 0.05-0.08 is good)

Example

python src/main.py train --model-path-base model

training log like this

epoch 1 batch 40 processed 640 batch-loss 0.1984 epoch-elapsed 0h00m10s total-elapsed 0h00m11s 
epoch 1 batch 41 processed 656 batch-loss 0.2119 epoch-elapsed 0h00m10s total-elapsed 0h00m11s 
epoch 1 batch 42 processed 672 batch-loss 0.2205 epoch-elapsed 0h00m11s total-elapsed 0h00m11s 
epoch 1 batch 43 processed 688 batch-loss 0.2128 epoch-elapsed 0h00m11s total-elapsed 0h00m11s 
# Dev Average Loss: 0.212 (MSE) -> 0.461 (RMSD)

Note

Please use GPU/CUDA configuration to run this block

To test

python src/main.py test --model-path-base [saved_model_directory]

This will evaluate a trained model. Testing log will be outputed to test_log.txt by default.

Example

1. if you have blind testset to predict:
   python src/main.py test --model-path-base model-0/model_dev\=0.0772/
2. if you have full testset (with label) to predict and validate:
   python src/main.py test_withlabel --model-path-base model-0/model_dev\=0.0772/
   Outpit like this:                

   # python3 src/main.py test_withlabel --model-path-base model-0/model_dev=0.0772Loading data...Loading model...initializing vacabularies... done.Sequence(6): ['<START>', '<STOP>', 'A', 'C', 'G', 'U']Brackets(5): ['<START>', '<STOP>', '(', ')', '.']W0113 21:57:44.871776   221 device_context.cc:252] Please NOTE: device: 0, CUDA Capability: 70, Driver API Version: 11.0, Runtime API Version: 9.0W0113 21:57:44.878015   221 device_context.cc:260] device: 0, cuDNN Version: 7.6.#  Dev Average Loss: 0.0772 (MSE) -> 0.2778 (RMSD)# Test Average Loss: 0.0445 (MSE) -> 0.2111 (RMSD)

   登录后复制
              

• Since we don't offer labelled testset in baseline, you may not run test_withlabel unless you create ~/data/test.txt by yourself.

Note

Please use GPU/CUDA configuration to run this block

One more thing

You can continue to improve on the basis of the baseline model, or use a brand new model. You can try to tune from the following aspects:

• Preprocess input data to extract more features. The baseline model uses the RNA secondary structure predicted from LinearFold as parts of features. You can try to add more features with other secondary structure prediction software (Vienna RNAfold, RNAstructure, CONTRAfold, etc.)
• More complex embedding
• Use pre-trained models such as Elmo and Bert in the Embedding layer
• Optimize network structure and parameters
  • Hidden layer size selection-width and number of layers
  • Try complex network architecture
  • Try regularization, dropout, etc. to avoid overfitting
  • Optimize hyperparameters such as learning rate
  • Choose a good loss function
  • Try different optimizers