自然语言处理NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 SRL

本文转载自查看原文 2021-11-04 12:50 141

# Gavin大咖金句

Gavin：理论上将Transformer能够更好的处理一切以“set of units”存在的数据，而计算机视觉、语音、自然语言处理等属于这种类型的数据，所以理论上讲Transformer会在接下来数十年对这些领域形成主导性的统治力。

Gavin：A feedforward network with a single layer is sufficient to represent any function, but the layer may be infeasibly large and may fail to learn and generalize correctly. — Ian Goodfellow, DLB

Gavin：Transformer是人工智能领域的新一代的引擎，本质是研究结构关系、工业界实践的核心是基于Transformer实现万物皆流。

Gavin：Non-linearity是Transformer的魔法

在本文中，我们将运行一个基于BERT的Transformer解决问题的示例，让我们从一个复杂的样本开始，基于BERT变换器分析这个样本

样本4将我们带入更棘手的SRL领域，该示例将“Alice”与动词“liked”分开，形成了一种长期依赖关系，必须跳过“whose
husband went jogging every Sunday.”

句子的样本为：

"Alice, whose husband went jogging every Sunday, liked to go to a dancing class in the meantime."

人类可以隔离“Alice”，并找到谓词：

"Alice, whose husband went jogging every Sunday, liked to go to a dancing class in the meantime."

BERT模型能像我们一样找到谓词吗？让我们首先运行SRL.ipynb中的代码

!echo '{"sentence": "Alice, whose husband went jogging every Sunday, liked to go to a dancing class in the meantime."}' | \
allennlp predict https://storage.googleapis.com/allennlp-public-models/bert-base-srl-2020.03.24.tar.gz -

运行结果如下：

2020-12-20 09:08:43,153 - INFO - transformers.file_utils - PyTorch version 1.5.1 available.
2020-12-20 09:08:43.324358: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.10.1
2020-12-20 09:08:45,080 - INFO - transformers.file_utils - TensorFlow version 2.4.0 available.
2020-12-20 09:08:46,294 - INFO - allennlp.common.file_utils - checking cache for https://storage.googleapis.com/allennlp-public-models/bert-base-srl-2020.03.24.tar.gz at /root/.allennlp/cache/e20d5b792a8d456a1a61da245d1856d4b7778efe69ac3c30759af61940aa0f42.f72523a9682cb1f5ad3ecf834075fe53a1c25a6bcbf4b40c11e13b7f426a4724
2020-12-20 09:08:46,294 - INFO - allennlp.common.file_utils - waiting to acquire lock on /root/.allennlp/cache/e20d5b792a8d456a1a61da245d1856d4b7778efe69ac3c30759af61940aa0f42.f72523a9682cb1f5ad3ecf834075fe53a1c25a6bcbf4b40c11e13b7f426a4724
2020-12-20 09:08:46,295 - INFO - filelock - Lock 139693451663232 acquired on /root/.allennlp/cache/e20d5b792a8d456a1a61da245d1856d4b7778efe69ac3c30759af61940aa0f42.f72523a9682cb1f5ad3ecf834075fe53a1c25a6bcbf4b40c11e13b7f426a4724.lock
2020-12-20 09:08:46,295 - INFO - allennlp.common.file_utils - cache of https://storage.googleapis.com/allennlp-public-models/bert-base-srl-2020.03.24.tar.gz is up-to-date
2020-12-20 09:08:46,295 - INFO - filelock - Lock 139693451663232 released on /root/.allennlp/cache/e20d5b792a8d456a1a61da245d1856d4b7778efe69ac3c30759af61940aa0f42.f72523a9682cb1f5ad3ecf834075fe53a1c25a6bcbf4b40c11e13b7f426a4724.lock
2020-12-20 09:08:46,295 - INFO - allennlp.models.archival - loading archive file https://storage.googleapis.com/allennlp-public-models/bert-base-srl-2020.03.24.tar.gz from cache at /root/.allennlp/cache/e20d5b792a8d456a1a61da245d1856d4b7778efe69ac3c30759af61940aa0f42.f72523a9682cb1f5ad3ecf834075fe53a1c25a6bcbf4b40c11e13b7f426a4724
2020-12-20 09:08:46,295 - INFO - allennlp.models.archival - extracting archive file /root/.allennlp/cache/e20d5b792a8d456a1a61da245d1856d4b7778efe69ac3c30759af61940aa0f42.f72523a9682cb1f5ad3ecf834075fe53a1c25a6bcbf4b40c11e13b7f426a4724 to temp dir /tmp/tmpmnefy_l0
2020-12-20 09:08:50,552 - INFO - allennlp.common.params - type = from_instances
2020-12-20 09:08:50,552 - INFO - allennlp.data.vocabulary - Loading token dictionary from /tmp/tmpmnefy_l0/vocabulary.
2020-12-20 09:08:50,552 - INFO - filelock - Lock 139692801159064 acquired on /tmp/tmpmnefy_l0/vocabulary/.lock
2020-12-20 09:08:50,580 - INFO - filelock - Lock 139692801159064 released on /tmp/tmpmnefy_l0/vocabulary/.lock
2020-12-20 09:08:50,580 - INFO - allennlp.common.params - model.type = srl_bert
2020-12-20 09:08:50,581 - INFO - allennlp.common.params - model.regularizer = None
2020-12-20 09:08:50,581 - INFO - allennlp.common.params - model.bert_model = bert-base-uncased
2020-12-20 09:08:50,581 - INFO - allennlp.common.params - model.embedding_dropout = 0.1
2020-12-20 09:08:50,581 - INFO - allennlp.common.params - model.initializer = <allennlp.nn.initializers.InitializerApplicator object at 0x7f0cc3c6a7b8>
2020-12-20 09:08:50,581 - INFO - allennlp.common.params - model.label_smoothing = None
2020-12-20 09:08:50,581 - INFO - allennlp.common.params - model.ignore_span_metric = False
2020-12-20 09:08:50,581 - INFO - allennlp.common.params - model.srl_eval_path = /usr/local/lib/python3.6/dist-packages/allennlp_models/structured_prediction/tools/srl-eval.pl
2020-12-20 09:08:50,888 - INFO - transformers.configuration_utils - loading configuration file https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-uncased-config.json from cache at /root/.cache/torch/transformers/4dad0251492946e18ac39290fcfe91b89d370fee250efe9521476438fe8ca185.7156163d5fdc189c3016baca0775ffce230789d7fa2a42ef516483e4ca884517
2020-12-20 09:08:50,889 - INFO - transformers.configuration_utils - Model config BertConfig {
  "architectures": [
    "BertForMaskedLM"
  ],
  "attention_probs_dropout_prob": 0.1,
  "hidden_act": "gelu",
  "hidden_dropout_prob": 0.1,
  "hidden_size": 768,
  "initializer_range": 0.02,
  "intermediate_size": 3072,
  "layer_norm_eps": 1e-12,
  "max_position_embeddings": 512,
  "model_type": "bert",
  "num_attention_heads": 12,
  "num_hidden_layers": 12,
  "pad_token_id": 0,
  "type_vocab_size": 2,
  "vocab_size": 30522
}

2020-12-20 09:08:50,928 - INFO - transformers.modeling_utils - loading weights file https://cdn.huggingface.co/bert-base-uncased-pytorch_model.bin from cache at /root/.cache/torch/transformers/f2ee78bdd635b758cc0a12352586868bef80e47401abe4c4fcc3832421e7338b.36ca03ab34a1a5d5fa7bc3d03d55c4fa650fed07220e2eeebc06ce58d0e9a157
2020-12-20 09:08:53,601 - INFO - allennlp.nn.initializers - Initializing parameters
2020-12-20 09:08:53,602 - INFO - allennlp.nn.initializers - Done initializing parameters; the following parameters are using their default initialization from their code
2020-12-20 09:08:53,602 - INFO - allennlp.nn.initializers -    bert_model.embeddings.LayerNorm.bias
2020-12-20 09:08:53,602 - INFO - allennlp.nn.initializers -    bert_model.embeddings.LayerNorm.weight
2020-12-20 09:08:53,602 - INFO - allennlp.nn.initializers -    bert_model.embeddings.position_embeddings.weight
2020-12-20 09:08:53,602 - INFO - allennlp.nn.initializers -    bert_model.embeddings.token_type_embeddings.weight
2020-12-20 09:08:53,602 - INFO - allennlp.nn.initializers -    bert_model.embeddings.word_embeddings.weight
2020-12-20 09:08:53,602 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.0.attention.output.LayerNorm.bias
2020-12-20 09:08:53,602 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.0.attention.output.LayerNorm.weight
2020-12-20 09:08:53,602 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.0.attention.output.dense.bias
2020-12-20 09:08:53,602 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.0.attention.output.dense.weight
2020-12-20 09:08:53,602 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.0.attention.self.key.bias
2020-12-20 09:08:53,602 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.0.attention.self.key.weight
2020-12-20 09:08:53,602 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.0.attention.self.query.bias
2020-12-20 09:08:53,603 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.0.attention.self.query.weight
2020-12-20 09:08:53,603 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.0.attention.self.value.bias
2020-12-20 09:08:53,603 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.0.attention.self.value.weight
2020-12-20 09:08:53,603 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.0.intermediate.dense.bias
2020-12-20 09:08:53,603 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.0.intermediate.dense.weight
2020-12-20 09:08:53,603 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.0.output.LayerNorm.bias
2020-12-20 09:08:53,603 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.0.output.LayerNorm.weight
2020-12-20 09:08:53,603 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.0.output.dense.bias
2020-12-20 09:08:53,603 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.0.output.dense.weight
2020-12-20 09:08:53,603 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.1.attention.output.LayerNorm.bias
2020-12-20 09:08:53,603 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.1.attention.output.LayerNorm.weight
2020-12-20 09:08:53,603 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.1.attention.output.dense.bias
2020-12-20 09:08:53,603 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.1.attention.output.dense.weight
2020-12-20 09:08:53,603 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.1.attention.self.key.bias
2020-12-20 09:08:53,603 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.1.attention.self.key.weight
2020-12-20 09:08:53,603 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.1.attention.self.query.bias
2020-12-20 09:08:53,603 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.1.attention.self.query.weight
2020-12-20 09:08:53,603 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.1.attention.self.value.bias
2020-12-20 09:08:53,603 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.1.attention.self.value.weight
2020-12-20 09:08:53,603 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.1.intermediate.dense.bias
2020-12-20 09:08:53,603 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.1.intermediate.dense.weight
2020-12-20 09:08:53,603 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.1.output.LayerNorm.bias
2020-12-20 09:08:53,604 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.1.output.LayerNorm.weight
2020-12-20 09:08:53,604 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.1.output.dense.bias
2020-12-20 09:08:53,604 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.1.output.dense.weight
2020-12-20 09:08:53,604 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.10.attention.output.LayerNorm.bias
2020-12-20 09:08:53,604 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.10.attention.output.LayerNorm.weight
2020-12-20 09:08:53,604 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.10.attention.output.dense.bias
2020-12-20 09:08:53,604 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.10.attention.output.dense.weight
2020-12-20 09:08:53,604 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.10.attention.self.key.bias
2020-12-20 09:08:53,604 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.10.attention.self.key.weight
2020-12-20 09:08:53,604 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.10.attention.self.query.bias
2020-12-20 09:08:53,604 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.10.attention.self.query.weight
2020-12-20 09:08:53,604 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.10.attention.self.value.bias
2020-12-20 09:08:53,604 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.10.attention.self.value.weight
2020-12-20 09:08:53,604 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.10.intermediate.dense.bias
2020-12-20 09:08:53,604 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.10.intermediate.dense.weight
2020-12-20 09:08:53,604 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.10.output.LayerNorm.bias
2020-12-20 09:08:53,604 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.10.output.LayerNorm.weight
2020-12-20 09:08:53,604 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.10.output.dense.bias
2020-12-20 09:08:53,604 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.10.output.dense.weight
2020-12-20 09:08:53,604 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.11.attention.output.LayerNorm.bias
2020-12-20 09:08:53,604 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.11.attention.output.LayerNorm.weight
2020-12-20 09:08:53,604 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.11.attention.output.dense.bias
2020-12-20 09:08:53,604 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.11.attention.output.dense.weight
2020-12-20 09:08:53,605 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.11.attention.self.key.bias
2020-12-20 09:08:53,605 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.11.attention.self.key.weight
2020-12-20 09:08:53,605 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.11.attention.self.query.bias
2020-12-20 09:08:53,605 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.11.attention.self.query.weight
2020-12-20 09:08:53,605 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.11.attention.self.value.bias
2020-12-20 09:08:53,605 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.11.attention.self.value.weight
2020-12-20 09:08:53,605 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.11.intermediate.dense.bias
2020-12-20 09:08:53,605 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.11.intermediate.dense.weight
2020-12-20 09:08:53,605 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.11.output.LayerNorm.bias
2020-12-20 09:08:53,605 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.11.output.LayerNorm.weight
2020-12-20 09:08:53,605 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.11.output.dense.bias
2020-12-20 09:08:53,605 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.11.output.dense.weight
2020-12-20 09:08:53,605 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.2.attention.output.LayerNorm.bias
2020-12-20 09:08:53,605 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.2.attention.output.LayerNorm.weight
2020-12-20 09:08:53,605 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.2.attention.output.dense.bias
2020-12-20 09:08:53,605 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.2.attention.output.dense.weight
2020-12-20 09:08:53,605 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.2.attention.self.key.bias
2020-12-20 09:08:53,605 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.2.attention.self.key.weight
2020-12-20 09:08:53,605 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.2.attention.self.query.bias
2020-12-20 09:08:53,605 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.2.attention.self.query.weight
2020-12-20 09:08:53,605 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.2.attention.self.value.bias
2020-12-20 09:08:53,605 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.2.attention.self.value.weight
2020-12-20 09:08:53,606 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.2.intermediate.dense.bias
2020-12-20 09:08:53,606 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.2.intermediate.dense.weight
2020-12-20 09:08:53,606 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.2.output.LayerNorm.bias
2020-12-20 09:08:53,606 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.2.output.LayerNorm.weight
2020-12-20 09:08:53,606 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.2.output.dense.bias
2020-12-20 09:08:53,606 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.2.output.dense.weight
2020-12-20 09:08:53,606 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.3.attention.output.LayerNorm.bias
2020-12-20 09:08:53,606 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.3.attention.output.LayerNorm.weight
2020-12-20 09:08:53,606 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.3.attention.output.dense.bias
2020-12-20 09:08:53,606 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.3.attention.output.dense.weight
2020-12-20 09:08:53,606 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.3.attention.self.key.bias
2020-12-20 09:08:53,606 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.3.attention.self.key.weight
2020-12-20 09:08:53,606 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.3.attention.self.query.bias
2020-12-20 09:08:53,606 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.3.attention.self.query.weight
2020-12-20 09:08:53,606 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.3.attention.self.value.bias
2020-12-20 09:08:53,606 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.3.attention.self.value.weight
2020-12-20 09:08:53,606 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.3.intermediate.dense.bias
2020-12-20 09:08:53,606 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.3.intermediate.dense.weight
2020-12-20 09:08:53,606 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.3.output.LayerNorm.bias
2020-12-20 09:08:53,606 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.3.output.LayerNorm.weight
2020-12-20 09:08:53,606 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.3.output.dense.bias
2020-12-20 09:08:53,606 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.3.output.dense.weight
2020-12-20 09:08:53,607 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.4.attention.output.LayerNorm.bias
2020-12-20 09:08:53,607 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.4.attention.output.LayerNorm.weight
2020-12-20 09:08:53,607 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.4.attention.output.dense.bias
2020-12-20 09:08:53,607 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.4.attention.output.dense.weight
2020-12-20 09:08:53,607 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.4.attention.self.key.bias
2020-12-20 09:08:53,607 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.4.attention.self.key.weight
2020-12-20 09:08:53,607 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.4.attention.self.query.bias
2020-12-20 09:08:53,607 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.4.attention.self.query.weight
2020-12-20 09:08:53,607 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.4.attention.self.value.bias
2020-12-20 09:08:53,607 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.4.attention.self.value.weight
2020-12-20 09:08:53,607 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.4.intermediate.dense.bias
2020-12-20 09:08:53,607 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.4.intermediate.dense.weight
2020-12-20 09:08:53,607 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.4.output.LayerNorm.bias
2020-12-20 09:08:53,607 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.4.output.LayerNorm.weight
2020-12-20 09:08:53,607 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.4.output.dense.bias
2020-12-20 09:08:53,607 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.4.output.dense.weight
2020-12-20 09:08:53,607 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.5.attention.output.LayerNorm.bias
2020-12-20 09:08:53,607 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.5.attention.output.LayerNorm.weight
2020-12-20 09:08:53,607 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.5.attention.output.dense.bias
2020-12-20 09:08:53,607 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.5.attention.output.dense.weight
2020-12-20 09:08:53,607 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.5.attention.self.key.bias
2020-12-20 09:08:53,607 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.5.attention.self.key.weight
2020-12-20 09:08:53,607 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.5.attention.self.query.bias
2020-12-20 09:08:53,608 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.5.attention.self.query.weight
2020-12-20 09:08:53,608 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.5.attention.self.value.bias
2020-12-20 09:08:53,608 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.5.attention.self.value.weight
2020-12-20 09:08:53,608 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.5.intermediate.dense.bias
2020-12-20 09:08:53,608 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.5.intermediate.dense.weight
2020-12-20 09:08:53,608 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.5.output.LayerNorm.bias
2020-12-20 09:08:53,608 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.5.output.LayerNorm.weight
2020-12-20 09:08:53,608 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.5.output.dense.bias
2020-12-20 09:08:53,608 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.5.output.dense.weight
2020-12-20 09:08:53,608 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.6.attention.output.LayerNorm.bias
2020-12-20 09:08:53,608 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.6.attention.output.LayerNorm.weight
2020-12-20 09:08:53,608 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.6.attention.output.dense.bias
2020-12-20 09:08:53,608 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.6.attention.output.dense.weight
2020-12-20 09:08:53,608 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.6.attention.self.key.bias
2020-12-20 09:08:53,608 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.6.attention.self.key.weight
2020-12-20 09:08:53,608 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.6.attention.self.query.bias
2020-12-20 09:08:53,608 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.6.attention.self.query.weight
2020-12-20 09:08:53,608 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.6.attention.self.value.bias
2020-12-20 09:08:53,608 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.6.attention.self.value.weight
2020-12-20 09:08:53,608 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.6.intermediate.dense.bias
2020-12-20 09:08:53,608 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.6.intermediate.dense.weight
2020-12-20 09:08:53,608 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.6.output.LayerNorm.bias
2020-12-20 09:08:53,608 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.6.output.LayerNorm.weight
2020-12-20 09:08:53,609 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.6.output.dense.bias
2020-12-20 09:08:53,609 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.6.output.dense.weight
2020-12-20 09:08:53,609 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.7.attention.output.LayerNorm.bias
2020-12-20 09:08:53,609 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.7.attention.output.LayerNorm.weight
2020-12-20 09:08:53,609 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.7.attention.output.dense.bias
2020-12-20 09:08:53,609 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.7.attention.output.dense.weight
2020-12-20 09:08:53,609 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.7.attention.self.key.bias
2020-12-20 09:08:53,609 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.7.attention.self.key.weight
2020-12-20 09:08:53,609 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.7.attention.self.query.bias
2020-12-20 09:08:53,609 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.7.attention.self.query.weight
2020-12-20 09:08:53,609 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.7.attention.self.value.bias
2020-12-20 09:08:53,683 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.7.attention.self.value.weight
2020-12-20 09:08:53,683 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.7.intermediate.dense.bias
2020-12-20 09:08:53,683 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.7.intermediate.dense.weight
2020-12-20 09:08:53,683 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.7.output.LayerNorm.bias
2020-12-20 09:08:53,683 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.7.output.LayerNorm.weight
2020-12-20 09:08:53,683 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.7.output.dense.bias
2020-12-20 09:08:53,683 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.7.output.dense.weight
2020-12-20 09:08:53,683 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.8.attention.output.LayerNorm.bias
2020-12-20 09:08:53,684 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.8.attention.output.LayerNorm.weight
2020-12-20 09:08:53,684 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.8.attention.output.dense.bias
2020-12-20 09:08:53,684 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.8.attention.output.dense.weight
2020-12-20 09:08:53,684 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.8.attention.self.key.bias
2020-12-20 09:08:53,684 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.8.attention.self.key.weight
2020-12-20 09:08:53,684 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.8.attention.self.query.bias
2020-12-20 09:08:53,684 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.8.attention.self.query.weight
2020-12-20 09:08:53,684 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.8.attention.self.value.bias
2020-12-20 09:08:53,684 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.8.attention.self.value.weight
2020-12-20 09:08:53,684 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.8.intermediate.dense.bias
2020-12-20 09:08:53,684 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.8.intermediate.dense.weight
2020-12-20 09:08:53,684 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.8.output.LayerNorm.bias
2020-12-20 09:08:53,684 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.8.output.LayerNorm.weight
2020-12-20 09:08:53,684 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.8.output.dense.bias
2020-12-20 09:08:53,684 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.8.output.dense.weight
2020-12-20 09:08:53,684 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.9.attention.output.LayerNorm.bias
2020-12-20 09:08:53,685 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.9.attention.output.LayerNorm.weight
2020-12-20 09:08:53,685 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.9.attention.output.dense.bias
2020-12-20 09:08:53,685 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.9.attention.output.dense.weight
2020-12-20 09:08:53,685 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.9.attention.self.key.bias
2020-12-20 09:08:53,685 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.9.attention.self.key.weight
2020-12-20 09:08:53,685 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.9.attention.self.query.bias
2020-12-20 09:08:53,685 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.9.attention.self.query.weight
2020-12-20 09:08:53,685 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.9.attention.self.value.bias
2020-12-20 09:08:53,685 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.9.attention.self.value.weight
2020-12-20 09:08:53,685 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.9.intermediate.dense.bias
2020-12-20 09:08:53,685 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.9.intermediate.dense.weight
2020-12-20 09:08:53,685 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.9.output.LayerNorm.bias
2020-12-20 09:08:53,685 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.9.output.LayerNorm.weight
2020-12-20 09:08:53,685 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.9.output.dense.bias
2020-12-20 09:08:53,685 - INFO - allennlp.nn.initializers -    bert_model.encoder.layer.9.output.dense.weight
2020-12-20 09:08:53,685 - INFO - allennlp.nn.initializers -    bert_model.pooler.dense.bias
2020-12-20 09:08:53,685 - INFO - allennlp.nn.initializers -    bert_model.pooler.dense.weight
2020-12-20 09:08:53,686 - INFO - allennlp.nn.initializers -    tag_projection_layer.bias
2020-12-20 09:08:53,686 - INFO - allennlp.nn.initializers -    tag_projection_layer.weight
2020-12-20 09:08:54,186 - INFO - allennlp.common.params - dataset_reader.type = srl
2020-12-20 09:08:54,187 - INFO - allennlp.common.params - dataset_reader.lazy = False
2020-12-20 09:08:54,187 - INFO - allennlp.common.params - dataset_reader.cache_directory = None
2020-12-20 09:08:54,187 - INFO - allennlp.common.params - dataset_reader.max_instances = None
2020-12-20 09:08:54,187 - INFO - allennlp.common.params - dataset_reader.manual_distributed_sharding = False
2020-12-20 09:08:54,187 - INFO - allennlp.common.params - dataset_reader.manual_multi_process_sharding = False
2020-12-20 09:08:54,187 - INFO - allennlp.common.params - dataset_reader.token_indexers = None
2020-12-20 09:08:54,187 - INFO - allennlp.common.params - dataset_reader.domain_identifier = None
2020-12-20 09:08:54,187 - INFO - allennlp.common.params - dataset_reader.bert_model_name = bert-base-uncased
2020-12-20 09:08:54,497 - INFO - transformers.tokenization_utils - loading file https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-uncased-vocab.txt from cache at /root/.cache/torch/transformers/26bc1ad6c0ac742e9b52263248f6d0f00068293b33709fae12320c0e35ccfbbb.542ce4285a40d23a559526243235df47c5f75c197f04f37d1a0c124c32c9a084
input 0:  {"sentence": "Alice, whose husband went jogging every Sunday, liked to go to a dancing class in the meantime."}
prediction:  {"verbs": [{"verb": "went", "description": "Alice , [ARG1: whose husband] [V: went] [ARG2: jogging] [ARGM-TMP: every Sunday] , liked to go to a dancing class in the meantime .", "tags": ["O", "O", "B-ARG1", "I-ARG1", "B-V", "B-ARG2", "B-ARGM-TMP", "I-ARGM-TMP", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O"]}, {"verb": "jogging", "description": "Alice , [ARG0: whose husband] went [V: jogging] [ARGM-TMP: every Sunday] , liked to go to a dancing class in the meantime .", "tags": ["O", "O", "B-ARG0", "I-ARG0", "O", "B-V", "B-ARGM-TMP", "I-ARGM-TMP", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O"]}, {"verb": "liked", "description": "[ARG0: Alice , whose husband went jogging every Sunday] , [V: liked] [ARG1: to go to a dancing class in the meantime] .", "tags": ["B-ARG0", "I-ARG0", "I-ARG0", "I-ARG0", "I-ARG0", "I-ARG0", "I-ARG0", "I-ARG0", "O", "B-V", "B-ARG1", "I-ARG1", "I-ARG1", "I-ARG1", "I-ARG1", "I-ARG1", "I-ARG1", "I-ARG1", "I-ARG1", "O"]}, {"verb": "go", "description": "[ARG0: Alice , whose husband went jogging every Sunday] , liked to [V: go] [ARG4: to a dancing class] [ARGM-TMP: in the meantime] .", "tags": ["B-ARG0", "I-ARG0", "I-ARG0", "I-ARG0", "I-ARG0", "I-ARG0", "I-ARG0", "I-ARG0", "O", "O", "O", "B-V", "B-ARG4", "I-ARG4", "I-ARG4", "I-ARG4", "B-ARGM-TMP", "I-ARGM-TMP", "I-ARGM-TMP", "O"]}, {"verb": "dancing", "description": "Alice , whose husband went jogging every Sunday , liked to go to a [V: dancing] [ARG0: class] in the meantime .", "tags": ["O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "B-V", "B-ARG0", "O", "O", "O", "O"]}], "words": ["Alice", ",", "whose", "husband", "went", "jogging", "every", "Sunday", ",", "liked", "to", "go", "to", "a", "dancing", "class", "in", "the", "meantime", "."]}

2020-12-20 09:08:55,842 - INFO - allennlp.models.archival - removing temporary unarchived model dir at /tmp/tmpmnefy_l0

原始输出相当长，有详细的描述，让我们关注感兴趣的部分，看看模型是否找到谓词。是的！它发现了动词“like”。如原始输出的摘录所示：

[ARG0: Alice , whose husband went jogging every Sunday] , [V: liked]

现在让我们看看运行AllenNLP在线用户界面上的示例。transformer首先找到了Alice's husband：

Transformer 解释为：
- 谓语或动词为“went”

- “whose husband”是一个参数

- “jogging”是与“went”相关的另一个参数

- “every Sunday”是原始输出中表示为[ARGM-TMP: every Sunday]

Transformer 发现了Alice's husband was doing

我们可以看到，动词“jogging”被识别，并与“whose husband”有关，用“every Sunday”的时态修饰，Transformer 并没有停在那里，它现在可以检测出Alice 喜欢什么

描述Alice的参数有点长，但十分正确。如果我们回到SRL.ipynb的输出，我们可以看到原始详细信息证实了分析是正确的：

[ARG0: Alice , whose husband went jogging every Sunday] , [V: liked]
[ARG1: to go to a dancing class in the meantime]

transformer 还正确检测和分析动词“go”：

我们可以看到，“in the meantime”的时间修饰也被识别，当我们想到BERT模型训练时使用的简单序列+动词输入时，这是一个相当好的表现。最后，transformer将最后一个动词“dancing”标识为与“class”相关：

现在我们来看看AllenNLP在线用户界面生成的格式化文本输出

input 0: {
    "sentence": "Alice, whose husband went jogging every Sunday, liked to go to a dancing class in the meantime."
}
prediction: {
    "verbs": [{
        "verb": "went",
        "description": "Alice , [ARG1: whose husband] [V: went] [ARG2: jogging] [ARGM-TMP: every Sunday] , liked to go to a dancing class in the meantime .",
        "tags": ["O", "O", "B-ARG1", "I-ARG1", "B-V", "B-ARG2", "B-ARGM-TMP", "I-ARGM-TMP", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O"]
    }, {
        "verb": "jogging",
        "description": "Alice , [ARG0: whose husband] went [V: jogging] [ARGM-TMP: every Sunday] , liked to go to a dancing class in the meantime .",
        "tags": ["O", "O", "B-ARG0", "I-ARG0", "O", "B-V", "B-ARGM-TMP", "I-ARGM-TMP", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O"]
    }, {
        "verb": "liked",
        "description": "[ARG0: Alice , whose husband went jogging every Sunday] , [V: liked] [ARG1: to go to a dancing class in the meantime] .",
        "tags": ["B-ARG0", "I-ARG0", "I-ARG0", "I-ARG0", "I-ARG0", "I-ARG0", "I-ARG0", "I-ARG0", "O", "B-V", "B-ARG1", "I-ARG1", "I-ARG1", "I-ARG1", "I-ARG1", "I-ARG1", "I-ARG1", "I-ARG1", "I-ARG1", "O"]
    }, {
        "verb": "go",
        "description": "[ARG0: Alice , whose husband went jogging every Sunday] , liked to [V: go] [ARG4: to a dancing class] [ARGM-TMP: in the meantime] .",
        "tags": ["B-ARG0", "I-ARG0", "I-ARG0", "I-ARG0", "I-ARG0", "I-ARG0", "I-ARG0", "I-ARG0", "O", "O", "O", "B-V", "B-ARG4", "I-ARG4", "I-ARG4", "I-ARG4", "B-ARGM-TMP", "I-ARGM-TMP", "I-ARGM-TMP", "O"]
    }, {
        "verb": "dancing",
        "description": "Alice , whose husband went jogging every Sunday , liked to go to a [V: dancing] [ARG0: class] in the meantime .",
        "tags": ["O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "B-V", "B-ARG0", "O", "O", "O", "O"]
    }],
    "words": ["Alice", ",", "whose", "husband", "went", "jogging", "every", "Sunday", ",", "liked", "to", "go", "to", "a", "dancing", "class", "in", "the", "meantime", "."]
}

# 星空智能对话机器人系列博客
- [NLP星空智能对话机器人系列：第二次星空智能对话机器人Zoom线上演示安排](https://blog.csdn.net/duan_zhihua/article/details/119973461?spm=1001.2014.3001.5501)
- [NLP星空智能对话机器人系列：StarSpace: Embed All The Things](https://blog.csdn.net/duan_zhihua/article/details/120069995?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：Facebook StarSpace框架初体验](https://blog.csdn.net/duan_zhihua/article/details/120117492?spm=1001.2014.3001.5501)
- [NLP星空智能对话机器人系列：Facebook StarSpace框架案例数据加载](https://blog.csdn.net/duan_zhihua/article/details/120121272?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理多头注意力架构-通过Python实例计算Q, K, V](https://blog.csdn.net/duan_zhihua/article/details/120230267?spm=1001.2014.3001.5501)
- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理多头注意力架构 Q K V注意力评分](https://blog.csdn.net/duan_zhihua/article/details/120232738?spm=1001.2014.3001.5502)
- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理多头注意力架构 Concatenation of the output of the heads](https://blog.csdn.net/duan_zhihua/article/details/120234517?spm=1001.2014.3001.5501)
- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理位置编码（positional_encoding）](https://blog.csdn.net/duan_zhihua/article/details/120256529?spm=1001.2014.3001.5501)
- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理位置编码案例应用](https://blog.csdn.net/duan_zhihua/article/details/120262856?spm=1001.2014.3001.5501)
- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理基于BERT模型微调实现句子分类](https://blog.csdn.net/duan_zhihua/article/details/120316808?spm=1001.2014.3001.5501)
- [NLP星空智能对话机器人系列：第4、5、6、7次星空智能对话机器人Zoom线上演示安排](https://blog.csdn.net/duan_zhihua/article/details/120382427?spm=1001.2014.3001.5501)
- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 KantaiBERT](https://blog.csdn.net/duan_zhihua/article/details/120373424?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 KantaiBERT ByteLevelBPETokenizer](https://blog.csdn.net/duan_zhihua/article/details/120384508?spm=1001.2014.3001.5501)
- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 KantaiBERT Initializing model](https://blog.csdn.net/duan_zhihua/article/details/120392929?spm=1001.2014.3001.5501)
- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 KantaiBERT Exploring the parameters](https://blog.csdn.net/duan_zhihua/article/details/120399404?spm=1001.2014.3001.5501)
- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 KantaiBERT Initializing the trainer](https://duanzhihua.blog.csdn.net/article/details/120405419)
- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 KantaiBERT Language modeling with FillMaskPipeline](https://blog.csdn.net/duan_zhihua/article/details/120405815?spm=1001.2014.3001.5501)
- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 GLUE CoLA](https://blog.csdn.net/duan_zhihua/article/details/120422365?spm=1001.2014.3001.5501)
- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 GLUE SST-2](https://blog.csdn.net/duan_zhihua/article/details/120441864?spm=1001.2014.3001.5501)
- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 GLUE MRPC](https://blog.csdn.net/duan_zhihua/article/details/120443327?spm=1001.2014.3001.5501)
- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 GLUE Winograd schemas and NER](https://blog.csdn.net/duan_zhihua/article/details/120473728?spm=1001.2014.3001.5501)
- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 Workshop on Machine Translation (WMT)](https://duanzhihua.blog.csdn.net/article/details/120495610)
- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 WMT数据处理](https://blog.csdn.net/duan_zhihua/article/details/120516215?spm=1001.2014.3001.5501)
- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 BLEU smoothing](https://blog.csdn.net/duan_zhihua/article/details/120565514?spm=1001.2014.3001.5501)
- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 Translations with Trax](https://blog.csdn.net/duan_zhihua/article/details/120572401?spm=1001.2014.3001.5501)
- [NLP星空智能对话机器人系列：第五次星空智能对话机器人线上演示](https://blog.csdn.net/duan_zhihua/article/details/120583102?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 BertViz](https://blog.csdn.net/duan_zhihua/article/details/120607908?spm=1001.2014.3001.5501)
- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 The Reformer](https://blog.csdn.net/duan_zhihua/article/details/120610502?spm=1001.2014.3001.5501)
- [NLP星空智能对话机器人系列：重磅信息：10月份一共会安排6次星空对话的机器人线上演示交流](https://blog.csdn.net/duan_zhihua/article/details/120615817?spm=1001.2014.3001.5501)
- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 Pattern-Exploiting Training (PET)](https://blog.csdn.net/duan_zhihua/article/details/120615259?spm=1001.2014.3001.5501)
- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 The philosophy of Pattern-Exploiting Training (PET)](https://blog.csdn.net/duan_zhihua/article/details/120618581?spm=1001.2014.3001.5501)
- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 It‘s time to make a decision](https://blog.csdn.net/duan_zhihua/article/details/120625482?spm=1001.2014.3001.5501)
- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 Text completion with GPT-2](https://blog.csdn.net/duan_zhihua/article/details/120629841?spm=1001.2014.3001.5501)
- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 Text completion with GPT-2 step3-5](https://blog.csdn.net/duan_zhihua/article/details/120632856?spm=1001.2014.3001.5501)
- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 Text completion with GPT-2 step 6-8](https://blog.csdn.net/duan_zhihua/article/details/120640746?spm=1001.2014.3001.5501)
- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 Text completion with GPT-2 step 9](https://blog.csdn.net/duan_zhihua/article/details/120658569?spm=1001.2014.3001.5501)
- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 Training a GPT-2 language model](https://blog.csdn.net/duan_zhihua/article/details/120678518?spm=1001.2014.3001.5501)
- [NLP星空智能对话机器人系列：论文学习 Do Transformers Really Perform Bad for Graph Representation](https://blog.csdn.net/duan_zhihua/article/details/120688601?spm=1001.2014.3001.5501)
- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 Training a GPT-2 language model Steps 2 to 6](https://blog.csdn.net/duan_zhihua/article/details/120700685?spm=1001.2014.3001.5501)
- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 Training a GPT-2 language model Steps 7 to 9](https://blog.csdn.net/duan_zhihua/article/details/120710206?spm=1001.2014.3001.5501)
- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 Training a GPT-2 language model Steps 10](https://blog.csdn.net/duan_zhihua/article/details/120720980?spm=1001.2014.3001.5501)
- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 T5-large transformer model](https://blog.csdn.net/duan_zhihua/article/details/120751601?spm=1001.2014.3001.5501)
- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 Architecture of the T5 model](https://blog.csdn.net/duan_zhihua/article/details/120761331?spm=1001.2014.3001.5501)
- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 Summarizing documents with T5-large](https://blog.csdn.net/duan_zhihua/article/details/120770732?spm=1001.2014.3001.5502)
- [NLP星空智能对话机器人系列：业界动态信息分享自然语言处理中的小样本学习](https://blog.csdn.net/duan_zhihua/article/details/120772216?spm=1001.2014.3001.5501)
- [NLP星空智能对话机器人系列：业界动态信息分享 Transformer 的性能瓶颈分析与优化](https://blog.csdn.net/duan_zhihua/article/details/120790068?spm=1001.2014.3001.5501)
- [自然语言处理NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 Matching datasets and tokenizers](https://blog.csdn.net/duan_zhihua/article/details/120793814?spm=1001.2014.3001.5501)
- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 Noisy relationships](https://blog.csdn.net/duan_zhihua/article/details/120816733?spm=1001.2014.3001.5501)
- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 Standard NLP tasks with specific vocabulary](https://blog.csdn.net/duan_zhihua/article/details/120835204?spm=1001.2014.3001.5501)
- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 T5 Bill of Rights Sample](https://blog.csdn.net/duan_zhihua/article/details/120903287?spm=1001.2014.3001.5501)
- [NLP星空智能对话机器人系列：论文解读 How Good is Your Tokenizer? (你的词元分析器有多好？多语言模型的单语性能研究)](https://blog.csdn.net/duan_zhihua/article/details/120949735?spm=1001.2014.3001.5501)
- [NLP星空智能对话机器人系列： Faiss Tutorial](https://blog.csdn.net/duan_zhihua/article/details/121053075?spm=1001.2014.3001.5501)
- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 Semantic Role Labeling (SRL).](https://blog.csdn.net/duan_zhihua/article/details/121006165?spm=1001.2014.3001.5501)
- [NLP星空智能对话机器人系列：Gavin大咖 Transformer 101问答集第一问](https://blog.csdn.net/duan_zhihua/article/details/121059773?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 SRL Sample 1](https://blog.csdn.net/duan_zhihua/article/details/121097719?spm=1001.2014.3001.5501)
- [自然语言处理NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 SRL(Semantic Role Labeling) Sample 2](https://zhuanlan.zhihu.com/p/428883118)
- [自然语言处理NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 SRL(Semantic Role Labeling)](https://www.jianshu.com/p/636a35838d0c)

免责声明！

本站转载的文章为个人学习借鉴使用，本站对版权不负任何法律责任。如果侵犯了您的隐私权益，请联系本站邮箱yoyou2525@163.com删除。

猜您在找 深入理解Transformer自然语言处理 SRL(Semantic Role Labeling) NLP自然语言处理 NLP 自然语言处理之综述学习笔记TF059:自然语言处理、智能聊天机器人 NLP 自然语言处理实战自然语言处理(NLP)——简介 SLAM+语音机器人DIY系列：（七）语音交互与自然语言处理——3.自然语言处理云计算引擎 Python自然语言处理-系列一 Pytorch系列:（六）自然语言处理NLP 【自然语言处理篇】--Chatterbot聊天机器人