๐Ÿ“• CS224n Lecture 11 ConvNets for NLP

11๊ฐ•! ๋งˆ์ง€๋ง‰ ๊ณผ์ œ์ธ ๊ณผ์ œ 5๋„ ์Šฌ์Šฌ ๋งˆ๊ฐ์œผ๋กœ ๋ณด์ธ๋‹ค. ๊ฐ•์˜์—์„œ๋„ second half๋ผ๊ณ  ํ•œ๋‹ค! ์ด์ œ๋ถ€ํ„ฐ ๊ฑฐ์˜ ์†Œ๊ฐœ์— ๊ฐ€๊น๋‹ค๊ณ  ํ•œ๋‹ค.

Suggested Readings. ๋‚˜์ค‘์— ์ฝ์–ด๋ด์•ผ์ง€

  1. Convolutional Neural Networks for Sentence Classification
  2. A Convolutional Neural Network for Modelling Sentences

์ด๊ฑด ์ฝ์œผ๋ฉด ์ข‹๋‹ค๋Š” ์ฑ…

Natural language processing with PyTorch : build intelligent language applications using deep learning

๊ฐ•์˜ ์ดˆ๋ฐ˜์— CNN์— ๊ด€ํ•œ ๊ฐ„๋žตํ•œ ์„ค๋ช…์ด ๋‚˜์™€์žˆ๋Š”๋ฐ, ์ด๊ฒƒ์€ CS231n์—์„œ ๋” ์ž์„ธํ•˜๊ฒŒ ์•Œ๋ ค์ค€๋‹ค. ํ•˜์ง€๋งŒ ํ•ด๋‹น ๊ฐ•์˜ ์ดˆ๋ฐ˜์„ ์ด๋ฏธ ๋“ค์–ด์„œ ์ญ‰ ๋„˜๊ธฐ๋ฉด์„œ ๋“ค์—ˆ๋‹ค.

Why CNNs?

CNN์„ ์‚ฌ์šฉํ•˜๊ธฐ ์œ„ํ•ด์„œ RNN๊ณผ ๊ตฌ๋ถ„๋˜๋Š” CNN์˜ ์žฅ์ ๊ณผ RNN์˜ ๋‹จ์ ์„ ์•Œ์•„๋ณด์ž.

์šฐ์„  RNN์€ phrase๋ฅผ prefix context ์—†์ด ์žก์•„๋‚ด์ง€ ๋ชปํ•œํ•˜๊ณ , phrase๋ฅผ ์žก์•„๋‚ผ ๋•Œ ๋‹จ์–ด๋ฅผ ๋„ˆ๋ฌด ๋งŽ์ด ์žก์•„๋‚ธ๋‹ค.

ํ•˜์ง€๋งŒ CNN์€ ํŠน์ •ํ•œ ๊ธธ์ด์˜ word subsequence๋ฅผ ๋ชจ๋‘ ๋งŒ๋“ค์–ด ๊ณ„์‚ฐํ•˜๋ฏ€๋กœ, ๋ฌธ๋ฒ•์ ์œผ๋กœ ์˜ณ์€ phrase๋งŒ์„ ์žก์•„๋‚ด๋Š” ๊ฒƒ์ด ์•„๋‹ˆ๋‹ค.

Single Layer CNN for Sentence Classification

Sentence Classification์— ๊ด€ํ•œ Yoon Kim (2014)์˜ ๋…ผ๋ฌธ์„ ์ฐธ๊ณ ํ•˜๋ฉด ์ข‹๋‹ค๊ณ  ํ•œ๋‹ค. ํ•ด๋‹น ๋…ผ๋ฌธ์˜ ์ฝ”๋“œ๋Š” github yoonkim/CNN_sentence์— ์žˆ๋‹ค.

CNN์„ sentence classification์— ํ™œ์šฉํ•˜๊ธฐ ์œ„ํ•ด์„œ ์‚ฌ์šฉํ•œ๋‹ค. ์ฃผ๋กœ sentiment ๋ถ„์„์„ ์œ„ํ•œ ์šฉ๋„๋กœ ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ๋‹ค๊ณ  ํ•œ๋‹ค. A Sensitivity Analysis of (and Practitionersโ€™ Guide to) Convolutional Neural Networks for Sentence Classification ๋„ ๋‚˜์ค‘์— ์ฝ์–ด๋ณด์ž.

๊ฐ•์˜์—์„œ ์ถ”๊ฐ€๋กœ ์ข€ ๋” ์‚ดํŽด๋ณผ ์ˆ˜ ์žˆ๋Š” ๋‚ด์šฉ, ํ‚ค์›Œ๋“œ๋กœ ๋‚˜์˜จ ๊ฒƒ์€ โ€œMultiple filter๋ฅผ ์ด์šฉํ•˜๋ฉด ์–ด๋–จ๊นŒ?โ€, โ€œMultiple Channel์„ ์ด์šฉํ•˜๋ฉด ์–ด๋–จ๊นŒ?โ€, Dropout, BatchNorm, 1x1 convolution ๋“ฑ์ด๋‹ค. ์•„๋ž˜๋Š” ๊ทธ ์ƒ์„ธํ•œ ๋‚ด์šฉ + ์ถ”๊ฐ€ ๋งํฌ

์—ฌํŠผ RNN์€ ๋Š๋ฆฌ๊ณ , ๊ทธ๋ž˜์„œ ๋” ๋‹ค์–‘ํ•œ ๋ฐฉ๋ฒ•์„ ์ฐพ๋Š”๋‹ค.

June 8, 2019 ์— ์ž‘์„ฑ
Tags: cs224n machine learning nlp