bert_question_answering_demo

OpenVINO 2020.4から今、AI業界で話題のBERTのデモが入るようになりましたので、早速試してみます

その前にBERTによるquestion answering というものはなんでしょうか?
分かりやすいというか、そのもののページがこちらです。
と書いてしまうと簡単なのですが、ごくごく簡単に説明すると対象となるWebサイトの情報を元に質問すると答えを見つけてくれるデモと書いた方が良いでしょうか
まだまだ私の理解もそれほどではないので、これくらいしか説明できませんが…

https://openvino.jp/configuration-ubuntu1804/
を参考にOpenVINO 2020.4をインストールしてください

bert_question_answering_demoは
/opt/intel/openvino/deployment_tools/open_model_zoo/demos/python_demos/bert_question_answering_demo
に格納されています

$ cd opt/intel/openvino/deployment_tools/open_model_zoo/demos/python_demos/bert_question_answering_demo

$ ls
README.md  bert_question_answering_demo.py  models.lst  requirements.txt

$ python3 /opt/intel/openvino/deployment_tools/open_model_zoo/tools/downloader/downloader.py --list models.lst -o ~/openvino_models/

ここまででモデルがダウンロードできると思います
ただし、私の環境では、bert-large-uncased-whole-word-masking-squad-fp32-0001でFAILEDが出てしまいました
そのため、他のモデルで試すことにします

$ ./bert_question_answering_demo.py -h
 usage: bert_question_answering_demo.py [-h] -v VOCAB -m MODEL -i INPUT
                                        [--input_names INPUT_NAMES]
                                        [--output_names OUTPUT_NAMES]
                                        [--model_squad_ver MODEL_SQUAD_VER]
                                        [-q MAX_QUESTION_TOKEN_NUM]
                                        [-a MAX_ANSWER_TOKEN_NUM] [-d DEVICE]
                                        [-r] [-c]
 Options:
   -h, --help            Show this help message and exit.
   -v VOCAB, --vocab VOCAB
                         Required. path to the vocabulary file with tokens
   -m MODEL, --model MODEL
                         Required. Path to an .xml file with a trained model
   -i INPUT, --input INPUT
                         Required. URL to a page with context
   --input_names INPUT_NAMES
                         Optional. Inputs names for the network. Default values
                         are "input_ids,attention_mask,token_type_ids"
   --output_names OUTPUT_NAMES
                         Optional. Outputs names for the network. Default
                         values are "output_s,output_e"
   --model_squad_ver MODEL_SQUAD_VER
                         Optional. SQUAD version used for model fine tuning
   -q MAX_QUESTION_TOKEN_NUM, --max_question_token_num MAX_QUESTION_TOKEN_NUM
                         Optional. Maximum number of tokens in question
   -a MAX_ANSWER_TOKEN_NUM, --max_answer_token_num MAX_ANSWER_TOKEN_NUM
                         Optional. Maximum number of tokens in answer
   -d DEVICE, --device DEVICE
                         Optional. Target device to perform inference
                         on.Default value is CPU
   -r, --reshape         Optional. Auto reshape sequence length to the input
                         context + max question len (to improve the speed)
   -c, --colors          Optional. Nice coloring of the questions/answers.
                         Might not work on some terminals (like Windows* cmd
                         console)

VOCAB ファイルとモデルファイルが必須になります
vocabulary fileはopen model zooフォルダの中に格納されています
モデルは先ほどダウンロードしたものを使用します

$ ./bert_question_answering_demo.py -v ../../../models/intel/bert-large-uncased-whole-word-masking-squad-int8-0001/vocab.txt -m ~/openvino_models/intel/bert-large-uncased-whole-word-masking-squad-int8-0001/FP16-INT8/bert-large-uncased-whole-word-masking-squad-int8-0001.xml -c --input="https://en.wikipedia.org/wiki/Bert_(Sesame_Street)"

上記コマンドでセサミストリートのBertにアクセスします
ただし残念ながらこのままだと下記のようなエラーが出ます

[ ERROR ]     Actual network input->output names: ['result.1', 'result.2', 'result.3']->['5211', '5212']

InputネームとOutputネームを設定し、下記のようなコマンドで再実行してみましょう

$ ./bert_question_answering_demo.py -v ../../../models/intel/bert-large-uncased-whole-word-masking-squad-int8-0001/vocab.txt -m ~/openvino_models/intel/bert-large-uncased-whole-word-masking-squad-int8-0001/FP16-INT8/bert-large-uncased-whole-word-masking-squad-int8-0001.xml --input="https://en.wikipedia.org/wiki/Bert_(Sesame_Street)" --input_names="result.1,result.2,result.3" --output_names="5211,5212" -c
$ python3 ./bert_question_answering_demo.py -v ../../../models/intel/bert-large-uncased-whole-word-masking-squad-int8-0001/vocab.txt -m ~/openvino_models/intel/bert-large-uncased-whole-word-masking-squad-int8-0001/FP16-INT8/bert-large-uncased-whole-word-masking-squad-int8-0001.xml --input="https://en.wikipedia.org/wiki/Bert_(Sesame_Street)" --input_names="result.1,result.2,result.3" --output_names="5211,5212" -c 
 [ INFO ] Loading vocab file:    ../../../models/intel/bert-large-uncased-whole-word-masking-squad-int8-0001/vocab.txt
 [ INFO ] 30522 tokens loaded
 [ INFO ] Get context from https://en.wikipedia.org/wiki/Bert_(Sesame_Street)
 [ INFO ] Size: 1265 chars
 [ INFO ] Context: Bert is a golden yellow Muppet character on the long running PBS and HBO children's television show Sesame Street. Bert was originally performed by Frank Oz.
 Since 1997, Muppeteer Eric Jacobson has been phased in as Bert's primary performer. Bert has also made cameo appearances within The Muppets franchise, including The Muppet Show, The Muppet Movie, and The Muppets Take Manhattan, in addition to an appearance with Ernie on The Flip Wilson Show on September 14, 1972, on which he sang "Clink, Clank", a song about noises.[2]
 Bert's age is unclear, but performer on Sesame Street Live, Taylor Morgan, has said that "I just kind of try to think like a six year old or a seven year old, because that's how old Bert is."[3] He and his roommate Ernie form a comic duo that is one of the program's centerpieces, with Bert acting as the world weary foil to Ernie's naive troublemaking.
 As the duo often sing in their skits, several albums were released containing studio recorded versions of their songs. Bert's best known song is "Doin' the Pigeon". He and Ernie both had their own video, The Best of Ernie and Bert, and their own album, Bert and Ernie's Greatest Hits. Only Ernie, however, has hit the U.S. Billboard Hot 100 Chart with his song "Rubber Duckie".
 [ INFO ] Size: 300 tokens
 [ INFO ] Initializing Inference Engine
 [ INFO ] Plugin version is 2.1.2020.4.0-359-21e092122f4-releases/2020/4
 [ INFO ] Loading network files:
     /home/klf/openvino_models/intel/bert-large-uncased-whole-word-masking-squad-int8-0001/FP16-INT8/bert-large-uncased-whole-word-masking-squad-int8-0001.xml
     /home/klf/openvino_models/intel/bert-large-uncased-whole-word-masking-squad-int8-0001/FP16-INT8/bert-large-uncased-whole-word-masking-squad-int8-0001.bin
 ./bert_question_answering_demo.py:216: DeprecationWarning: 'inputs' property of IENetwork class is deprecated. To access DataPtrs user need to use 'input_data' property of InputInfoPtr objects which can be accessed by 'input_info' property.
   if ie_encoder.inputs.keys() != set(input_names) or ie_encoder.outputs.keys() != set(output_names):
 [ INFO ] Loading model to the CPU
Type question (empty string to exit):

WikipediaのBertのページを読み込んでいます
この状態で、プログラムは入力待機状態になります
そこで、who is bertと入力してみると、下記のように回答が返ります

Type question (empty string to exit):who is bert

[ INFO ] Size: 3 tokens
[ INFO ] Sequence of length 384 is processed with 1.74 requests/sec (0.58 sec per request)
[ INFO ] The performance below is reported only for reference purposes, please use the benchmark_app tool (part of the OpenVINO samples) for any actual measurements.
[ INFO ] 1 requests of 384 length were processed in 0.58sec (0.58sec per request)
[ INFO ] ---answer: 0.44 Bert
[ INFO ]    Bert is a golden yellow Muppet character on the long running PBS and HBO children's television show Sesame Street
Type question (empty string to exit):

ということで、Bertはセサミストリートに出ているゴールデンイエローのぬいぐるみと表示されました

ここまでは、ReadMe通りに行ったので、次は別のものを使用してみます

wikipediaのサトウヤシを読み込む

$ ./bert_question_answering_demo.py -v ../../../models/intel/bert-large-uncased-whole-word-masking-squad-int8-0001/vocab.txt -m ~/openvino_models/intel/bert-large-uncased-whole-word-masking-squad-int8-0001/FP16-INT8/bert-large-uncased-whole-word-masking-squad-int8-0001.xml  --input_names="result.1,result.2,result.3" --output_names="5211,5212" -c --input="https://en.wikipedia.org/wiki/Arenga_pinnata"

サトウヤシが何かはよくわかりませんが、Arenga Pinnataというみたいです
こちらを読み込ませてみます

Type question (empty string to exit):how high is arenga pinnata
 [ INFO ] Size: 8 tokens
 [ INFO ] Sequence of length 384 is processed with 1.64 requests/sec (0.61 sec per request)
 [ INFO ] Sequence of length 384 is processed with 1.76 requests/sec (0.57 sec per request)
 [ INFO ] Sequence of length 384 is processed with 1.77 requests/sec (0.57 sec per request)
 [ INFO ] Sequence of length 384 is processed with 1.70 requests/sec (0.59 sec per request)
 [ INFO ] The performance below is reported only for reference purposes, please use the benchmark_app tool (part of the OpenVINO samples) for any actual measurements.
 [ INFO ] 4 requests of 384 length were processed in 2.34sec (0.58sec per request)
 [ INFO ] ---answer: 0.52 20 m tall
 [ INFO ]    It is a medium-sized palm, growing to 20 m tall, with the trunk remaining covered by the rough old leaf bases
 [ INFO ] ---answer: 0.00 Arenga pinnata in the northwestern Philippines
 [ INFO ]     Irok is a local name for Arenga pinnata in the northwestern Philippines
 [ INFO ] ---answer: 0.00 Philippines
 [ INFO ]    Sugar (jaggery) is also commonly derived from the fresh sap in Indonesia and the Philippines
 Type question (empty string to exit):

いくつか回答が出ていますが、一番上位にあるものは20mの高さがあると表示されています
うまくいっているようですね

InputデータはURLで指定してますが、こちらは下記のようにwgetなどで得たデータを入れることも可能です

$ wget https://en.wikipedia.org/wiki/Arenga_pinnata -O /home/foo/Arenga_pinnata

$ ./bert_question_answering_demo.py -v ../../../models/intel/bert-large-uncased-whole-word-masking-squad-int8-0001/vocab.txt -m ~/openvino_models/intel/bert-large-uncased-whole-word-masking-squad-int8-0001/FP16-INT8/bert-large-uncased-whole-word-masking-squad-int8-0001.xml  --input_names="result.1,result.2,result.3" --output_names="5211,5212" -c --input="file:/home/foo/Arenga_pinnata"

次は日本語の処理をやってみましょう