๐Ÿ“• CS224n Lecture 10 (Textual) Question Answering

๋“œ๋””์–ด 10๊ฐ•์„ ์ •๋ฆฌํ•œ๋‹ค. ๊ธฐ๋ณธ์ ์œผ๋กœ QA์‹œ์Šคํ…œ์— ๊ด€ํ•œ ์„ค๋ช…์ด๋‹ค.

Motivation

๋ฐฉ๋Œ€ํ•œ ์–‘์˜ full-text documents์—์„œ ๋‹จ์ˆœํžˆ ๊ด€๋ จ์žˆ๋Š” ๋ฌธ์„œ๋“ค์„ ์ฐพ์•„๋‚ด๋Š” ๊ฒƒ์€ ํž˜๋“ค๋‹ค. ๊ทธ๋ฆฌ๊ณ  ๊ด€๋ จ์žˆ๋Š” ๋ฌธ์„œ๋ฅผ question์— ๋Œ€ํ•œ answer๋กœ ๋ฐ›๊ณ  ์‹ถ์–ดํ•œ๋‹ค.

์ด๊ฒƒ์„ ๋‘ ํŒŒํŠธ๋กœ ๋‚˜๋ˆ„์–ด๋ณด๋ฉด ์•„๋ž˜์™€ ๊ฐ™๋‹ค.

  1. finding document that might contain an answer
    • ์ด๊ฑด CS276์„ ์ฐธ๊ณ ํ•˜์ž
  2. finding answer in a paragraph or a document
    • ์ด๊ฑด Reading comprehension๊ณผ ๊ด€๋ จ์ด ์žˆ๊ณ , ์ด ๋ถ€๋ถ„์— ๋Œ€ํ•ด์„œ ์ด์ œ ์ˆ˜์—…ํ•œ๋‹ค๊ณ  ํ•œ๋‹ค.

Reading Comprehension

์ดˆ๊ธฐ์˜ NLP๋•Œ๋ถ€ํ„ฐ ์—ฐ๊ตฌ๋˜์–ด์˜ค๋‹ค๊ฐ€ 2013๋…„ MCTest1๋•Œ ์—„์ฒญ ํ™œ๋ฐœํ•˜๊ฒŒ ์—ฐ๊ตฌ๋˜์—ˆ๋‹ค๊ณ  ํ•œ๋‹ค. MCTest๊ฐ€ Machine Comprehension์— ๊ด€ํ•œ ๋Œ€ํšŒ์ธ ๊ฒƒ ๊ฐ™์€๋ฐ, Machine Comprehension์ด ์ฃผ์–ด์ง„ ํ…์ŠคํŠธ์— ๋Œ€ํ•ด ์งˆ๋ฌธ์ด ์ฃผ์–ด์ง€๋ฉด, ์ข‹์€ ๋‹ต์„ ๋‚ด์–ด๋†“๋Š” ๊ฒƒ์ด ์ฃผ์š” ํƒœ์Šคํฌ๋ผ๊ณ  ํ•œ๋‹ค.

Passage (P) + Question (Q) -> Answer (A)

SQuAD (Stanford Question Answering Dataset)2

QA ์‹œ์Šคํ…œ์„ ์œ„ํ•œ ์˜คํ”ˆ ๋ฐ์ดํ„ฐ์ด๊ณ , ํ•œ๋ฒˆ ๋‚˜์ค‘์— ์ž์„ธํžˆ ์‚ดํŽด๋ณด์•„์•ผ๊ฒ ๋‹ค. ํ•œ๊ตญ์–ด๋ฒ„์ „์œผ๋กœ๋Š” KorQuAD๊ฐ€ ์žˆ๋‹ค. 1.0, 1.1์— ๊ด€ํ•œ ๊ฐ„๋žตํ•œ ์„ค๋ช…์„ ํ•˜๊ณ  2.0์— ๋Œ€ํ•œ ์„ค๋ช…๋„ ํ•œ๋‹ค.

1.0์€ ๋‹ต์ด passage์•ˆ์— ๋ฌด์กฐ๊ฑด ์žˆ์—ˆ๊ณ , ์‹œ์Šคํ…œ์ด ํ›„๋ณด๋“ค์„ ๊ณ ๋ฅธ ๋‹ค์Œ์— ranking๋งŒ ํ•˜๋ฉด ๋˜์—ˆ๋‹ค. ๊ทธ๋ž˜์„œ ํ•ด๋‹น span์ด ๋‹ต์ธ์ง€ ์•„๋‹Œ์ง€๋ฅผ ํ™•์ธํ•  ํ•„์š”๊ฐ€ ์—†์—ˆ๋‹ค. ๊ทธ๋ž˜์„œ No Answer๊ฐ€ ์žˆ๋Š” ๊ฒƒ์„ ๋งŒ๋“ค์—ˆ๋‹ค๊ณ  ํ•œ๋‹ค.

SQuAD๋Š” ๋ฌด์กฐ๊ฑด span-based answer๋งŒ์„ ๊ฐ€์ ธ์˜ค๊ณ , question์ด ๋ฌด์กฐ๊ฑด passage๋ฅผ ์œ„ํ•ด์„œ ๊ตฌ์„ฑ๋œ ๊ฒƒ์ด๋ฉด์„œ, multi-fact/sentence inference๋Š” ๊ฑฐ์˜ ์—†๋‹ค๋Š” ์ ์ด๋‹ค. ๊ทธ๋ž˜๋„ well-targeted, well-structed, clean dataset์ด๋ผ๊ณ  ํ•œ๋‹ค. ๋‚˜์ค‘์— ํ•œ๋ฒˆ ํ† ์ด ํ”„๋กœ์ ํŠธ๋กœ ์‹œ๋„ํ•ด๋ณด์•„๋„ ์ข‹์„ ๋“ฏ ํ•˜๋‹ค.

Stanford Attentive Reader

์œ„ ๋…ผ๋ฌธ ๋‘๊ฐœ์™€ ๋‹ค๋ฅธ ํ•˜๋‚˜๊ฐ€ ๋” ์žˆ๋Š”๋ฐ [Chen 2018]์ด๋ผ๊ณ ๋งŒ ๋˜์–ด์žˆ์–ด์„œ ๋ญ”์ง€ ์ž˜ ๋ชจ๋ฅด๊ณ˜๋‹ค. ์ด๊ฑด ๋‚˜์ค‘์— ๊ฐ„๋‹จํ•˜๊ฒŒ ์ฝ์–ด๋ณด์ž. ์ž์‹ ๋“ค์˜ ํ•™๊ต์—์„œ ๋งŒ๋“  Reading Comprehension, QA ์‹œ์Šคํ…œ์„ ๋ณด์—ฌ์ฃผ๋Š” ๋“ฏ ํ•˜๋‹ค..

The Stanford Attentive Reader 1
The Stanford Attentive Reader 2

Stanford Attentive Reader++๋„ ์žˆ๋‹ค๊ณ  ํ•˜๋‹ˆ (์ด๊ฑด ๋ชจ๋ธ ๊ทธ๋ฆผ์ด ๋งŽ์ด ๋ณต์žกํ•ด๋ณด์ด๊ณ  ๊ฐ„๋‹จํ•œ ์ดํ•ด๊ฐ€ ๋˜์ง€ ์•Š์•„์„œ ๊ทธ๋ƒฅ ๋ฏธ์ฒจ๋ถ€) ๋‚˜์ค‘์— ๋” ์‚ดํŽด๋ณด์ž. (Chen et al., 2016; Chen et al., 2017)

BiDAF (Bi-Directional Attention Flow for Machine Comprehension) 3

Attention์„ ์–‘๋ฐฉํ–ฅ์œผ๋กœ ์‚ฌ์šฉํ•˜๊ธฐ ์œ„ํ•œ ๊ตฌ์กฐ์˜ ๋…ผ๋ฌธ. ๋ฉ”์ธ ์•„์ด๋””์–ด๋ฅผ โ€œthe Attention Flow layerโ€๋ผ๊ณ  ์ƒ๊ฐํ•˜๋ฉด ๋œ๋‹ค.

๊ทธ๋ฆฌ๊ณ  ๋˜ ๋‹ค๋ฅธ ๊ฒƒ๋“ค์€

๊ทธ๋ฆฌ๊ณ  ์ข€ ์ค‘์š”ํ•˜๊ฒŒ ๋” ์‚ดํŽด๋ณด๋ฉด ์ข‹์„ ๊ฒƒ

  • Elmo
  • Bert
  • SDNet : Bert๋ฅผ submodule๋กœ ์‚ฌ์šฉํ•œ ์—์ œ
  1. Link Machine Comprehension Testย 

  2. arxiv SQuAD์— ๊ด€ํ•œ ๋…ผ๋ฌธย 

  3. arxiv Seo, Kembhavi, Farhadi, Hajishirzi, ICLR 2017ย 

June 8, 2019 ์— ์ž‘์„ฑ
Tags: cs224n machine learning nlp