๐Ÿ“ƒ GPT ๋ฆฌ๋ทฐ

October 20, 2019

Transformer ๊ด€๋ จ์„ ์ฐพ์•„๋ณด๋ฉด์„œ Huggingface์˜ transformers ๋ ˆํฌ์ง€ํ† ๋ฆฌ์— BERT ๋‹ค์Œ์œผ๋กœ ๋‚˜์˜ค๋Š” OpenAI์˜ GPT๋ฅผ ์ฝ์–ด๋ณด๊ธฐ๋กœ ํ—€๋‹ค. ๋Œ€์‹  ์ •๋ฆฌ๊ฐ€ ๋„ˆ๋ฌด ์˜ค๋ž˜ ๊ฑธ๋ฆฌ๋Š” ๊ฒฐ๊ณผ ๊ฐ™์€ ๊ฑด ๋‹ค ์ œ์™ธํ•œ๋‹ค.

Tags: paper
Read More

๐Ÿ“ƒ BERT ๋ฆฌ๋ทฐ

October 14, 2019

์ตœ๊ทผ์— ๐Ÿค—/transformers์—์„œ ๋‹ค์–‘ํ•œ transformer based model์„ ์†Œ๊ฐœํ–ˆ๋Š”๋ฐ, ๊ทธ๋ž˜์„œ transformer ๊ธฐ๋ฐ˜์˜ ์—ฌ๋Ÿฌ ๋ชจ๋ธ๋“ค์„ ์ •๋ฆฌํ•ด๋ณด๋ ค ํ•œ๋‹ค. ์ฒซ๋ฒˆ์งธ๋กœ Google์˜ BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding๋ฅผ ์ •๋ฆฌํ•ด๋ณธ๋‹ค.

Tags: paper
Read More

๐Ÿ“ƒ Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data

October 7, 2019

MeCab์„ ํ•œ๊ตญ์–ด๋ฅผ ์œ„ํ•ด ์ƒˆ๋กœ ์ž‘์„ฑํ•ด๋ณด๊ณ  ์‹ถ์–ด์„œ ๊ทธ ๋…ผ๋ฌธ(Applying Conditional Random Fields to Japanese Morphological Analysis)์„ ์ฐพ์•„๋ณด์•˜๋”๋‹ˆ, CRF based segmentation์ด๋ผ๊ณ  ๋ถˆ๋Ÿฌ์„œ CRF์— ๋Œ€ํ•ด ์šฐ์„  ์ •๋ฆฌํ•œ๋‹ค. (ํ”„๋กœ ์•ผํฌ ์‰์ด๋ฒ„)

Tags: paper
Read More

๐ŸŽŠ Pycon KR 2019

September 3, 2019

ํŒŒ์ด์ฝ˜ KR 2019์— ๊ฐ€๋ณด๋ฉด์„œ ๋“ค์—ˆ๋˜ ๊ฝค ์ข‹์•˜๋˜ ์„ธ์…˜๋“ค ๋ชฉ๋ก + ๋‚ด์šฉ์„ ๊ฐ„๋‹จํ•˜๊ฒŒ ์ •๋ฆฌํ•˜๋ฉด์„œ ๊ทธ ๋‚ด์šฉ์„ ๋ธ”๋กœ๊ทธ์—๋„ ์˜ฎ๊ฒจ๋†“๋Š”๋‹ค. ํšŒ์‚ฌ๋ถ„๋“ค๊ป˜ ๊ณต์œ ํ•  ๋ชฉ์ ์œผ๋กœ ์ ์–ด๋†“์•„์„œ ๋งŽ์€ ๋ถ€๋ถ„์ด ๋น ์ ธ์žˆ๊ณ , ์—„์ฒญ ์ถ•์•ฝ๋˜์–ด ์žˆ๋Š” ๊ฒƒ๋„ ์žˆ๋‹ค.

Tags: conference
Read More

๐Ÿ“ƒ Review of "Attention Is All You Need"

July 21, 2019

Transformer๋ฅผ ์†Œ๊ฐœํ•˜๋Š” ๋…ผ๋ฌธ์œผ๋กœ, CS224n๊ฐ•์˜์˜ suggested readings ๋ชฉ๋ก์— ์žˆ์–ด์„œ ์ฝ์–ด๋ณธ ๋…ผ๋ฌธ์ด๋‹ค. ํ•œ๊ตญ์–ด ๋ฆฌ๋ทฐ๋„ ์—„์ฒญ ๋งŽ์„ ์ •๋„๋กœ ์œ ๋ช…ํ•œ ๋…ผ๋ฌธ์ด๋‹ค. ํ•ด๋‹น ๋…ผ๋ฌธ์„ ์ฝ๊ณ , ๊ฐ„๋žตํ•œ ์ •๋ฆฌ๋ฅผ ํ•ด๋ณด์•˜๋‹ค. ๋…ผ๋ฌธ์€ arXiv:1706.03762์— ์žˆ๋‹ค.

Tags: paper
Read More

๐Ÿ“• CS224n Lecture 14 Transformers and Self-Attention For Generative Models

June 9, 2019

14๊ฐ•์€ ๊ฐ•์—ฐ์ž๋ฅผ ์ดˆ๋Œ€ํ•ด์„œ ๊ฐ•์˜๋ฅผ ์ง„ํ–‰ํ•œ๋‹ค. Google AI์—์„œ ๋‚˜์˜จ ์—ฐ์‚ฌ์ž ๋‘๋ถ„์ด๋ผ๊ณ  ํ•œ๋‹ค. NLP ๊ณต๋ถ€ํ•˜๋ ค๊ณ  ๋“ฃ๋Š” ๊ฒƒ์ด๊ณ , ๋‹ค๋ฅธ ๊ฒƒ๋“ค ํ•˜๊ธฐ์—๋„ ์•ฝ๊ฐ„ ๋ฒ…์ฐฌ๋“ฏ ์‹ถ์–ด์„œ NLP ๋‚ด์šฉ์„ ์ข€ ๋ฒ—์–ด๋‚˜๋Š” ์ด๋ฏธ์ง€ ์ฒ˜๋ฆฌ, ์Œ์„ฑ ์ฒ˜๋ฆฌ ๊ฐ™์€...

Tags: cs224n
Read More