๐ŸŽŠ PyTorch Ecosystem Day 2021

PTED ๋ณด๋ฉด์„œ ๋‚ด์šฉ + ์‹ ๊ธฐํ•œ ๊ฒƒ๋“ค ๋ฉ”๋ชจ. ๋ถ€์Šค ํ˜•์‹์œผ๋กœ ์šด์˜๋˜์–ด์„œ ๋‚ด๊ฐ€ Gather Town์—์„œ ๋Œ์•„๋‹ค๋‹ˆ๋Š” ํ˜•์‹์ด์—ˆ๊ณ  ์ด์•ผ๊ธฐ๋„ ๋งŽ์ด ๋‚˜๋ˆŒ ์ˆ˜ ์žˆ์–ด์„œ ์ข‹์•˜๋‹ค.

ํ•œ๊ตญ์‹œ๊ฐ„์œผ๋กœ๋Š” 22์ผ ์ƒˆ๋ฒฝ์ด๊ณ , ๋ฏธ๊ตญ ์„œ๋ถ€ ๊ธฐ์ค€์œผ๋กœ 21์ผ ๋‚ฎ์— ์‹œ์ž‘๋˜์—ˆ๋‹ค. ์ธ์›์ˆ˜ ๋ณด๋‹ˆ๊นŒ ๋Œ€๋žต 300๋ช… ์ข€ ์•ˆ๋˜๊ฒŒ ์ฐธ๊ฐ€ํ•œ ๋Š๋‚Œ.

์‹œ์ž‘ ํ‚ค๋…ธํŠธ

์˜ˆ์ˆ˜ ํ˜•๋‹˜์˜ ์˜คํ”„๋‹์œผ๋กœ ์‹œ์ž‘ํ•œ๋‹ค!! ํฌ๋Ÿผ์—์„œ ํ™œ๋™ํ•˜๋‹ค๊ฐ€ ๊ทธ ์ดํ›„์— NVIDIA์—์„œ ๋ณธ๊ฒฉ์ ์œผ๋กœ ํŒŒ์ดํ† ์น˜ ๊ฐœ๋ฐœํ•˜๊ธฐ ์‹œ์ž‘ํ•˜์…จ๋‹ค๊ณ .. PyTorch ํฌ๋Ÿผ์—์„œ ํ™œ๋™์„ ๋งŽ์ด ํ•˜์‹œ๋Š” ๋ถ„์ด๋ผ ํŽ˜์ด์Šค๋ถ ๊ฒŒ์‹œ๊ธ€๋กœ ์˜คํ”„๋‹ ํ‚ค๋…ธํŠธ ์†Œ์‹์ด ์˜ฌ๋ผ์˜ค๋‹ˆ ์ €๋Ÿฐ ๋ฐ˜์‘๋„ ๋‚˜์˜จ๋‹ค.

https://www.facebook.com/pytorch/posts/2797012123933087

์•”ํŠผ ํ‚ค๋…ธํŠธ์—์„œ ๋งํ•˜๊ธธ, ํŒŒ์ดํ† ์น˜์˜ ์„ฑ๊ณต์š”์†Œ๋ฅผ ๊ผฝ์•„๋ณด๋ฉด flexibility, performance, community ์ •๋„.

PyTorch Release

๋ฒ„์ „์ด ๋นจ๋ฆฌ ์˜ฌ๋ผ๊ฐ„๋‹ค ์‹ถ์—ˆ๋Š”๋ฐ ๋ถ„๊ธฐ์— ํ•œ๋ฒˆ์”ฉ ๋ฉ”์ด์ € ๋ฒ„์ „์„ ์˜ฌ๋ฆฌ๋Š” ๊ฑฐ์˜€๊ตฌ๋‚˜. ๊ทธ๋ฆฌ๊ณ  Release Candidate๋„ ๊ธฐ๊ฐ„์ด ์ •ํ•ด์ ธ์žˆ๋‹ค๊ณ  ํ•œ๋‹ค. ๋ฆด๋ฆฌ์ฆˆ ํ•œ๋‹ฌ ์ „์— ๋ฏธ๋ฆฌ ๋‚ธ๋‹ค๊ณ . PyTorch ๋‚ด๋ถ€์—์„œ Feature๋Š” 3๊ฐ€์ง€๋กœ ๋ถ„๋ฅ˜๋œ๋‹ค๊ณ  ํ•œ๋‹ค. Stable, Beta, Prototype.

์•„๋ž˜๋Š” ์ข€ ์ƒˆ๋กœ์šด ๋ฆด๋ฆฌ์ฆˆ ์†Œ๊ฐœ

  • torch.fx: toolkit for developers to use to transform nn.Module
    • symbolic tracer, IR, python code generation์„ ํฌํ•จํ•œ๋‹ค.
    • ๋‚˜์ค‘์— ์ž์„ธํžˆ ์‚ดํŽด๋ณด์ž
  • beta] torch.linalg: np.linalg์™€ ๋˜‘๊ฐ™์ด ๋™์ž‘ํ•˜๊ฒŒ ํ•˜๋ ค๊ณ  ํ•œ๋‹ค.
    • torch.fft: ์ด๊ฒƒ๋„ numpy ํ˜ธํ™˜์„ฑ ๊ฐ™์€ ๋Š๋‚Œ์œผ๋กœ ๋งŒ๋“ค์–ด์กŒ๋‹ค๊ณ 
  • torch.profiler
    • GPU monitoring, Tensorboard plugin ์ถ”๊ฐ€
    • torch.autograd.profiler -> torch.profiler๋กœ ๋ณ€๊ฒฝ
    • ์ข‹๊ธด ํ•˜์ง€๋งŒ, ์‚ฌ์‹ค TensorFlow์—์„œ ์ž˜ ์ง€์›๋˜๋˜๊ฑฐ๋ผ ์•„์‰ฝ๊ธดํ•˜๋‹ค.
  • Distributed Training
    • beta] pipeline parallelism torch.distributed.pipeline.sync.Pipe
      • DDP๋งŒ์œผ๋กœ ์Šค์ผ€์ผ๋งํ•˜๊ธฐ ํž˜๋“œ๋‹ˆ๊นŒ..
      • Dev Day ๋‚ด์šฉ์ด๋ž‘ ๊ฒน์น˜๊ธด ํ•˜๋„ค
    • beta] DDP Communication Hook: ๊ธฐ๋ณธ all reduce๋ฅผ ์ˆ˜์ •ํ•ด์„œ ์“ธ ์ˆ˜ ์žˆ๋Š” ๊ธฐ๋Šฅ
  • beta] AMD Gpu binary

PyTorch Partner Collaborations

  • ์—ฌ๋Ÿฌ ๊ด€๋ จ ํšŒ์‚ฌ๋“ค ๋‚˜์™€์„œ ์†Œ๊ฐœํ•ด์ค€๋‹ค.
  • โ€ฆ ์Šคํ‚ต
  • ์ค‘๊ฐ„์— ๋ฐ๋ชจ๊ฐ€ ์ข€ ๋‚˜์˜ค๋Š”๋ฐ TensorBoard ์ƒ์˜ PyTorch Profiler ํ™”๋ฉด ์ž˜ ๋งŒ๋“ค์—ˆ๊ตฌ๋‚˜ ์‹ถ๋‹ค
  • ์™€ ๋””์ฆˆ๋‹ˆ์—์„œ๋„ ๋‚˜์™”์–ด์š”
    • ์˜ํ™” ์ œ์ž‘์—์„œ ๊ฒ€์ƒ‰์ด๋‚˜ ๋“ฑ๋“ฑ ์—ฌ๋Ÿฌ ์œ ํ‹ธ์„ฑ ๊ธฐ๋Šฅ์„ ML๋กœ ์ž˜ ํ•ด๊ฒฐํ–ˆ๋‹ค๋Š” ๋Š๋‚Œ์ธ๋ฐ, ์˜ˆ๋ฅผ ๋“ค์–ด ๋™์ผ ์บ๋ฆญํ„ฐ๊ฐ€ ๋‚˜์˜ค๋Š” ์”ฌ๋งŒ ๋‹ค ๊ธ์–ด๋ชจ์œผ๊ฑฐ๋‚˜ ํ•˜๋Š” ์‹
    • ๊ทธ๋ฆฌ๊ณ  ๋ฉ€ํ‹ฐ๋ชจ๋‹ฌ๋„ ๊ต‰์žฅํžˆ ์ž˜ ํ™œ์šฉํ•˜๊ณ  ์žˆ๋Š”๋ฐ, ๊ฑฐ๊ธฐ ๋ฐ์ดํ„ฐ๊ฐ€ ๋„ˆ๋ฌด ์ข‹์ž–์•„์š” ใ…‹ใ…‹ใ…‹ใ…‹ใ…‹

Community Updates

  • ํฌ๋Ÿผ์ด๋‚˜ GitHub ํ†ต๊ณ„๋ฅผ ๋ณด์—ฌ์ฃผ๋ฉด์„œ ๋งŽ์ด ์„ฑ์žฅํ–ˆ๋‹ค!๋ฅผ ๋ณด์—ฌ์คฌ๋‹ค.
  • Ecosystem์€ ์—ฌ๊ธฐ์—์„œ ํ™•์ธํ•˜์ž https://pytorch.org/ecosystem/
  • ์ด๋Ÿฐ ๊ฒƒ ๋ณผ ๋•Œ๋งˆ๋‹ค ์‰ฌ๋Š” ์‹œ๊ฐ„์— ์ปจํŠธ๋ฆฌ๋ทฐํŒ… ํ•˜๊ณ  ์‹ถ๋‹ค๋Š” ์ƒ๊ฐ์ด ๋งŽ์ด ๋“ ๋‹ค.

Posters

๋ณธ ํ–‰์‚ฌ. ํฌ์Šคํ„ฐ ์ด๋ฏธ์ง€๋ฅผ ๋ถ€์Šค๋งˆ๋‹ค ์ œ๊ณตํ•ด์คฌ๋Š”๋ฐ, PTD2์™€๋Š” ๋‹ค๋ฅด๊ฒŒ ๊ณต๊ฐœ๋˜์ง€ ์•Š๋‹ค๋ณด๋‹ˆ๊นŒ ์ด๋ฏธ์ง€๋ฅผ ์ฒจ๋ถ€ํ•˜๊ธด ํž˜๋“ค๋‹ค. ์•„๋ž˜๋Š” ๊ด€์‹ฌ์žˆ๋Š” ๋ถ€์Šค๋ฅผ ๋Œ์•„๋‹ค๋‹ˆ๋ฉด์„œ ๊ธฐ๋กํ•œ ๋‚ด์šฉ๋“ค์ด๋‹ค.

Compiler & Transform & Production

PyTorch development in VS Code

  • Microsoft VSCodeํŒ€์— ์†Œ์†๋œ ๋ถ„์ด ๋ถ€์Šค ์ง€ํ‚ค๊ณ  ์žˆ์—ˆ๋‹ค.
  • ์•„๋ž˜ ๊ธฐ๋Šฅ์€ ์ข‹์•„๋ณด์ธ๋‹ค
    • PyTorch Profiler integration
    • Tensorboard integration
    • Multi-dimensional Tensor data exploration
  • ๊ด€๋ จ ๋ธ”๋กœ๊ทธ ๊ธ€: https://devblogs.microsoft.com/python/python-in-visual-studio-code-february-2021-release/
  • TensorBoard plugin์€ ์จ๋ด์•ผ๊ฒ ๋‹ค.

Upcoming features in TorchScript

  • ํŽ˜๋ถ PyTorch Compiler Team
  • TorchScript Spec: https://fb.me/torchscript-spec
    • ๋‚˜์ค‘์— ์ฝ์–ด๋ณด์ž
  • Profile-directed Typing for TorchScript
    • ์ง์ ‘ ํƒ€์ž… ์จ์ฃผ๋Š” ๊ฒŒ ์‹œ๊ฐ„๋„ ๋˜๊ฒŒ ๋งŽ์ด ์“ฐ๊ณ , third-party๊ฐ€ ์ค‘๊ฐ„์— ์žˆ์–ด๋ฒ„๋ฆฌ๋ฉด ์—„์ฒญ ํž˜๋“ค๋‹ค.
    • ๊ทธ๋ž˜์„œ Profile-Directed Typing(PTD)๋ผ๋Š” ๊ธฐ๋Šฅ์„ ๊ณง ๋„ฃ์„๊ฑด๋ฐ, ์ด๊ฒŒ ๋ชจ๋ธ ์ฝ”๋“œ ๋‚ด์—์„œ ํ•จ์ˆ˜๋“ค์˜ ํƒ€์ž…์„ ๋‹ค ์žก์•„์ฃผ๋Š” ๊ฒƒ.
    • ์ง€๊ธˆ์€ Model Validity Check -> AST Construction -> IR Emission ์ˆœ์„œ์ธ๋ฐ, ์—ฌ๊ธฐ์— Model Execution -> Type Profiling -> Type Analysis ๊ณผ์ • ์ดํ›„ ๋‚˜์˜จ ํƒ€์ž…์„ IR๊ณผ ๊ฐ™์ด ๋‚ด๋ฑ‰์„ ์˜ˆ์ •์ธ ๊ฒƒ ๊ฐ™๋‹ค.
  • TorchScript Profiler
    • TorchScript๋Š” Performance Profiling์ด ํ•ญ์ƒ ์—†์—ˆ๋Š”๋ฐ, TorchScript-specific Profiler๋ฅผ ๋งŒ๋“œ๋Š” ์ค‘์ด๋ผ๊ณ  ํ•œ๋‹ค.
    • Python code mapping๋„ ๊ฐ€๋Šฅํ•˜๊ฒŒ ํ•  ์˜ˆ์ •
      • Code Mapping -> ์ด๊ฑฐ๋Š” ์ข‹๋‹ค!

PyTorch Quantization: FX Graph Mode Quantization

  • PyTorch Model Optimization Team
  • torch.fx ๋ชจ๋“ˆ์€ ์ง„์งœ ์—„์ฒญ ๋ฏธ๋Š” ๋Š๋‚Œ์ด๋‹ค.
    • TensorFlow์˜ Grappler ๊ธฐ๋Šฅ๊ณผ ๋น„์Šทํ•œ ๊ธฐ๋Šฅ์ด๋ผ๊ณ  ์ƒ๊ฐํ•˜๊ณ  ์žˆ๋Š”๋ฐ, TensorFlow๋Š” ์ด ๊ธฐ๋Šฅ์ด ๊ธฐ๋ณธ์ธ ๊ฒƒ์„ ๋ณด๋ฉด Graph ๋ชจ๋“œ์˜ ์žฅ์ ์ด ํฌ๊ธด ํ•˜๋‹ค๋ณด๋‹ค.
    • ๋‹ค๋งŒ ์•„์ง์€ Prototype ์ˆ˜์ค€์ด๋ผ๊ณ 
    • https://pytorch.org/docs/stable/quantization.html#quantization-api-summary Quantization ๋ฌธ์„œ๋ฅผ ๋ณด๋‹ˆ ์•„์ง์€ unstableํ•œ master ๋ฌธ์„œ๋ฅผ ๋ด๋‹ฌ๋ผ๊ณ  ํ•œ๋‹ค.
    • https://pytorch.org/docs/master/quantization.html#prototype-fx-graph-mode-quantization ์—ฌ๊ธฐ๋ฅผ ์ฐธ๊ณ ํ•˜๋ฉด ๋  ๋“ฏ
  • post training quantization์€ int8, float16 ๋œ๋‹ค๊ณ  ํ•˜๊ณ , QAT์—์„œ๋Š” int8 ํƒ€๊ฒŸ์ธ๊ฐ€ ๋ณด๋‹ค. fake quantization ๋ฐฉ์‹์œผ๋กœ ํ•˜๊ฒ ์ง€??
  • TensorFlow ์ชฝ์—์„œ Quantization ๊ด€๋ จ ๊ธฐ๋Šฅ์ด ์ƒ๋‹นํžˆ ์•„์‰ฌ์šด๋ฐ, Torch์ชฝ์ด ์ž˜ ๋œ๋‹ค๋ฉด ์˜คํžˆ๋ ค Onnx๋‚˜ TorchServe๋ฅผ ํ™œ์šฉํ•ด์„œ ์„œ๋น™ํ•˜๋Š”๊ฒŒ ๋” ๋น ๋ฅด๊ฒ ๋‹ค๋Š” ์ƒ๊ฐ์ด ๋“ ๋‹ค.
  • ํŽ˜์ด์Šค๋ถ ๋‚ด๋ถ€ ํ”„๋กœ๋•์…˜ ๋ชจ๋ธ์—๋Š” ์ด๋ฏธ ์ ์šฉ๋œ ์ƒํƒœ์ธ๋ฐ, ๋Œ€์ƒ ๋ชจ๋ธ์€ ๊ฑฐ์˜ vision ์ชฝ.
  • Next Steps
    • quantized graph๊ฐ€ ์•„์ง์€ ๋” ์ตœ์ ํ™”ํ•  ์—ฌ์ง€๊ฐ€ ์žˆ๋‹ค๊ณ  ํ•œ๋‹ค.
    • debug information์„ ์‹ ๊ฒฝ์“ธ ๊ฒƒ์ด๋ผ๊ณ  ํ•œ๋‹ค. -> ์ด๋Ÿฐ ๋ฐฉํ–ฅ์€ ์™„์ „ ์ฐฌ์„ฑ! ใ…‹ใ…‹

Accelerate deployment of deep learning models in production with Amazon EC2 Inf1 and TorchServe containers

  • ์ด๊ฑฐ ์˜ˆ์ „์— ์“ฐ๋ ค๊ณ  ํ•˜๋˜ ๊ธฐ๋Šฅ์ธ๋ฐ ๊ฒฐ๊ตญ ๋ชป์ผ์ง€๋งŒ, ์ด์ œ๋Š” ์ž˜ ๋˜๋‚˜๋ณด๋‹ค.
  • Inferentia chip์— ๋Œ€ํ•œ ๊ฐ„๋žตํ•œ ์„ค๋ช…์„ ๊ฐ€์ ธ์™€๋ณด์ž๋ฉด
    • fp16, bfloat16, int8๊ฐ™์€ ํƒ€์ž…๋“ค ์ถ”๋ก ๋„ ์ž˜ ์ง€์›ํ•˜๊ณ  mixed precision๋„ ๋œ๋‹ค.
    • 1 ~ 16 inferentia chip์„ ํ•œ ์ธ์Šคํ„ด์Šค์— ๋ฌผ๋ฆด ์ˆ˜ ์žˆ๋‹ค๊ณ  ํ•œ๋‹ค.
    • 4 NeuronCore๋กœ 128 TFlops ์ •๋„ ์„ฑ๋Šฅ
    • on-chip cache๊ฐ€ ๋งŽ์ด ๋‹ฌ๋ ค์žˆ๊ณ , 8GB DRAM ๋‹ฌ๋ ค์žˆ๋‹ค.
  • torch_neuron์ด๋ผ๋Š” ํŒจํ‚ค์ง€๋กœ ์—„์ฒญ ์‰ฝ๊ฒŒ ์‚ฌ์šฉ๊ฐ€๋Šฅํ•˜๋‹ค.
  • https://docs.aws.amazon.com/dlami/latest/devguide/tutorial-inferentia-pytorch-neuron.html ์ด๋Ÿฐ์‹์œผ๋กœ ์‚ฌ์šฉ

Torch.fx

  • FX: toolkit for writing Python-to-Python transforms
  • Symbolic Tracing
    • https://pytorch.org/docs/stable/fx.html ๋ฌธ์„œ ๋ดค๋Š”๋ฐ ์™„์ „ ์‹ ๊ธฐ..
    • ์•„๋ž˜ ์ฝ”๋“œ๋ณด๋ฉด ๋ฐ”๋กœ ์ดํ•ด๊ฐ€ ๊ฐ„๋‹ค.

      import torch
      # Simple module for demonstration
      class MyModule(torch.nn.Module):
          def __init__(self):
              super().__init__()
              self.param = torch.nn.Parameter(torch.rand(3, 4))
              self.linear = torch.nn.Linear(4, 5)
      
          def forward(self, x):
              return self.linear(x + self.param).clamp(min=0.0, max=1.0)
      
      module = MyModule()
      
      from torch.fx import symbolic_trace
      # Symbolic tracing frontend - captures the semantics of the module
      symbolic_traced : torch.fx.GraphModule = symbolic_trace(module)
      
      # High-level intermediate representation (IR) - Graph representation
      print(symbolic_traced.graph)
      """
      graph(x):
          %param : [#users=1] = self.param
          %add_1 : [#users=1] = call_function[target=<built-in function add>](args = (%x, %param), kwargs = {})
          %linear_1 : [#users=1] = call_module[target=linear](args = (%add_1,), kwargs = {})
          %clamp_1 : [#users=1] = call_method[target=clamp](args = (%linear_1,), kwargs = {min: 0.0, max: 1.0})
          return clamp_1
      """
      
      # Code generation - valid Python code
      print(symbolic_traced.code)
      """
      def forward(self, x):
          param = self.param
          add_1 = x + param;  x = param = None
          linear_1 = self.linear(add_1);  add_1 = None
          clamp_1 = linear_1.clamp(min = 0.0, max = 1.0);  linear_1 = None
          return clamp_1
      """
      
  • Graph-based Transformations
    • fx.Tracer, fx.Graph ์‚ฌ์šฉํ•˜๋ฉด ๋˜๊ฒŒ ํŽธํ•˜๊ฒŒ ๋ณ€๊ฒฝ๊ฐ€๋Šฅํ•˜๋‹ค๊ณ  ํ•œ๋‹ค.
    • AST ํŒŒ์‹ฑํ•˜๋“ฏ์ด ํŒŒ์‹ฑ์ด ๋˜๋„ค..?
    • โ€œgraph ํŒŒ์‹ฑ ํ›„ node๋ฅผ traverseํ•˜๋ฉด์„œ call_function ์ด๋ฉด์„œ torch.add ๋ผ๋ฉด torch.mul๋กœ Op ๋ณ€๊ฒฝโ€ ๊ฐ™์€ ์ผ์ด ๊ฐ€๋Šฅํ•˜๋‹ค.
    • -> Fused Operator ์ ์šฉ์ด ํ›จ์”ฌ ์‰ฌ์›Œ์งˆ ๋“ฏ
    • https://pytorch.org/docs/stable/fx.html#direct-graph-manipulation ๋ฌธ์„œ๋Š” ์—ฌ๊ธฐ ์„น์…˜ ์ฐธ๊ณ 
  • Python code generation
    • ์œ„ ์ฝ”๋“œ์—์„œ ๋ณด์ด๋“ฏ code generation์ด ๋ฐ”๋กœ ๋œ๋‹ค.
  • FX๊ฐ€ ์–ด๋”” ์“ฐ์ผ๊นŒ
    • ๊ณง ๋‚˜์˜ฌ graph-mode quantization์— ์“ฐ์ด๋Š” ์ค‘์ด๋‹ค.
    • ์ธํ…”์—์„œ ngraph(https://github.com/NervanaSystems/ngraph)๋ผ๋Š” ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ๋ฅผ ๊ฐœ๋ฐœํ•˜๋Š” ๊ฒƒ์œผ๋กœ ์•„๋Š”๋ฐ, ngraph ๊ฐ™์€ ๊ธฐ๋Šฅ์„ ๋ฐ”๋กœ ์ง€์›ํ•  ์ˆ˜๋„ ์žˆ๊ฒ ๋‹ค๋ผ๋Š” ์ƒ๊ฐ์ด ๋“ ๋‹ค.
  • ์˜ˆ์‹œ๋Š” https://github.com/pytorch/examples/tree/master/fx ์ฐธ๊ณ 

AI Model Efficiency Toolkit (AIMET)

  • Qualcomm์—์„œ ๊ฐœ๋ฐœํ•˜๋Š” ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ. https://github.com/quic/aimet
  • ๋‚˜์ค‘์— ๊ด€์‹ฌ์ƒ๊ธฐ๋ฉด ๋ด์•ผ์ง€ ใ…Žใ…Ž

Database & AI Accelerators

Enabling PyTorch on AMD Instinctโ„ข GPUs with the AMD ROCmโ„ข Open Software Platform

  • ๋˜๊ฒŒ ์˜ˆ์ „๋ถ€ํ„ฐ ์žˆ์—ˆ๋˜ ํ”„๋กœ์ ํŠธ๋กœ ๊ธฐ์–ตํ•˜๋Š”๋ฐ, โ€œM1์—์„œ ML ๋ชจ๋ธ ๋Œ๋ฆฌ๋Š” ๊ฒƒ ๊ฐ€๋Šฅํ•˜๋ƒ!!โ€๋ผ๋Š” ์–˜๊ธฐ๋‚˜์˜ค๋ฉด์„œ ๋ง‰ ๊ฐ™์ด ์†Œ์‹์ด ๋‚˜์™”๋˜ ๊ฑธ๋กœ ๊ธฐ์–ตํ•œ๋‹ค.
  • https://github.com/pytorch/pytorch/tree/master/torch/utils/hipify ์—ฌ๊ธฐ์„œ code conversionํ•˜๋Š” ๊ฒƒ ํ™•์ธ ๊ฐ€๋Šฅํ•˜๋‹ค.
  • ์ž์„ธํžˆ๋Š” ๊ด€์‹ฌ์—†์–ด์„œ ์—ฌ๊ธฐ๊นŒ์ง€๋งŒ..

Distributed Training

DeepSpeed: Shattering barriers of deep learning speed & scale

FairScale - A general purpose modular PyTorch library for high performance and large scale training

  • Facebook AI Research์—์„œ ๊ฐœ๋ฐœํ•˜๋Š” FairScale
  • torchgpipe, AdaScale, ZeRO, ZeRO-Offload ๋“ฑ๋“ฑ์„ ์ฐธ๊ณ ํ–ˆ๋Š”๋ฐ, ๊ทธ๊ฑธ ๋ณด์•„ DeepSpeed ๋“ฑ์—์„œ ๋งŽ์€ ๊ธฐ๋Šฅ์„ ๊ฐ€์ ธ์˜จ ๊ฒƒ์œผ๋กœ ๋ณด์ธ๋‹ค.
  • https://github.com/facebookresearch/fairscale
  • DeepSpeed๋Š” ์—ฌ๋Ÿฌ๋ชจ๋กœ ์‹œ๋„ํ–ˆ๋˜ ๋ถ„๋“ค์—๊ฒŒ ์•ฝ๊ฐ„์˜ ์žฅ๋ฒฝ์ด ์žˆ๋‹ค๊ณ  ๋“ค์—ˆ๋Š”๋ฐ, FairScale์€ ๊ทธ๋Ÿฐ ์ ์—์„œ ์ข€ ๋œํ•˜์ง€ ์•Š์„๊นŒ๋ผ๋Š” ๊ธฐ๋Œ€๊ฐ€ ์žˆ๋‹ค.
  • ํ•ต์‹ฌ ๊ธฐ๋Šฅ์€ ์•„๋ž˜์ •๋„
    • Pipeline Parallelism
    • AdaScale & Mixed Precision Training
    • Optimizer Sharding, Gradient Sharding, OffloadModel
  • ์‚ฌ์šฉ๋ฒ•์€ ํ™•์‹คํžˆ ๋” ๊ฐ„๋‹จํ•ด ๋ณด์ธ๋‹ค. -> Torch์ชฝ๊ณผ ์•„๋ฌด๋ž˜๋„ ๋” ๊ฐ€๊นŒ์šด ํšŒ์‚ฌ์ผํ…Œ๋‹ˆ ๊ธฐ๋Œ€ํ•ด๋ณผ ์ˆ˜ ์žˆ์ง€ ์•Š์„๊นŒ?

Accelerate PyTorch large model training with ONNX Runtime: just add one line of code

  • Microsoft AI
  • torch-ort๋ฅผ pip์œผ๋กœ ์„ค์น˜ํ›„ torch_ort.ORTModule๋กœ nn.Module์„ ๊ฐ์‹ธ์ฃผ๊ธฐ๋งŒ ํ•˜๋ฉด ๋˜๋Š” ๊ฐ„๋‹จํ•œ ์‚ฌ์šฉ๋ฒ•์ด๋‹ค.
  • DeepSpeed์™€ ์—ฐ๋™์ด ๊ฐ€๋Šฅํ•˜๋‹ค.
  • T5๋ฅผ ํ•™์Šตํ•  ๋•Œ ๋ฒค์น˜๋งˆํฌํ•ด๋ณด๋‹ˆ DeepSpeed ZeRO1 + ORTModule์„ ์‚ฌ์šฉํ•˜๋ฉด DeepSpeed ZeRO1๋งŒ ์‚ฌ์šฉํ–ˆ์„ ๋•Œ๋ณด๋‹ค 12%์ •๋„ ์„ฑ๋Šฅํ–ฅ์ƒ์ด ์žˆ๋‹ค๊ณ .
  • ์‚ฌ์šฉ๋ฒ•์ด ์—„์ฒญ ๊ฐ„๋‹จํ•ด์„œ ์จ๋ณผ๋งŒ ํ•  ๊ฒƒ ๊ฐ™๋‹ค.
  • ONNX๊ฐ€ ๊ทผ๋ฐ PyTorch ์ชฝ์ด๋ž‘ ์—„์ฒญ ๋ญ”๊ฐ€๋ฅผ ๋งŽ์ด ํ•˜๊ธด ํ•˜๋„ค์š”.

Frontend & Experiment Manager

Accelerate PyTorch with IPEX and oneDNN using Intel BF16 Technology

  • Intel, Facebook์ด ๊ฐ™์ด ์“ฐ์—ฌ์ ธ ์žˆ์—ˆ๋‹ค.
  • PyTorch ์ชฝ๋„ intel๊ณผ์˜ ํ˜‘๋ ฅ์œผ๋กœ oneAPI ํ†ตํ•ฉ์ด ๋˜์—ˆ๋‚˜๋ณด๋‹ค.
  • AVX512 & bfloat16 ์ง€์›์ด ๋ฉ”์ธ์ธ ๊ฒƒ์œผ๋กœ ๋ณด์ธ๋‹ค. (3์„ธ๋Œ€ ์ œ์˜จ ํ”„๋กœ์„ธ์„œ ๊ธฐ์ค€)
  • intel_pytorch_extension ํŒจํ‚ค์ง€ https://github.com/intel/intel-extension-for-pytorch
  • BERT Large ๊ธฐ์ค€ 1.41๋ฐฐ ์ •๋„ ์„ฑ๋Šฅํ–ฅ์ƒ

Hydra Framework

  • ์š”์ฆ˜ ๊ณ„์† ์“ฐ๋Š” ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ๋ผ ๊ธฐ๋ณธ ์‚ฌ์šฉ๋ฒ•์€ ํŒจ์Šค. ์ข‹๊ธด ์ง„์งœ ์ข‹๋‹ค.
    • ๊ฐ€๋” interpolation์ด๋‚˜ anchor๊ฐ™์€ ๊ธฐ๋Šฅ์ด ๋ง˜๋Œ€๋กœ ์•ˆ๋  ๋•Œ๋„ ์žˆ์ง€๋งŒ ๋นจ๋ฆฌ๋นจ๋ฆฌ ๊ณ ์ณ์ง€๋Š” ํŽธ
  • https://hydra.cc
  • Tab Completion์ด ๊ฐ€๋Šฅํ•œ์ง€ ๋ชฐ๋ž๋„ค..

NLP & Multimodal, RL & Time Series

Rolling out Transformers with TorchScript and Inferentia

  • Autodesk์—์„œ conversational bot์„ support์šฉ์œผ๋กœ ์šด์˜. ๊ทธ๊ฑธ Inferentia๋กœ ์ถ”๋ก 
  • BERT๋‚˜ ๊ทธ ๊ด€๋ จ ๋ชจ๋ธ๋กœ Sequence Classification Head ๋ถ™์—ฌ์„œ ํŒŒ์ธํŠœ๋‹.
  • AWS Inferentia, PyTorch๋กœ 4.9x ์„ฑ๋Šฅ ํ–ฅ์ƒ
  • Neuron SDK๋กœ ์ž˜ ๊ฐ์‹ธ๋ฉด ์—„์ฒญ ์ข‹์Œ

MMF: A modular framework for multimodal research

  • https://github.com/facebookresearch/mmf
  • Visual QA ๊ฐ™์€ ๊ฑฐ ์‰ฝ๊ฒŒ ํ•  ์ˆ˜ ์žˆ๊ฒ ๋‹ค.
  • Text, Image, Audio, Video ์ง€์›๋œ๋‹ค๊ณ .
  • ์‹œ๊ฐ„๋‚  ๋•Œ ์‚ดํŽด๋ด์•ผ๊ฒ ๋‹ค.

RL Based Performance Optimization of Deep Neural Networks

  • Facebook AI
  • ๊ธฐ์กด์—” Solution Space์—์„œ Candidates๋ฅผ ๊ณจ๋ผ์„œ Evaluation ํ•˜๋Š” ๊ฒƒ์„ ๋ฐ˜๋ณตํ•˜๋ฉด์„œ ์ตœ์ข… ์†”๋ฃจ์…˜์„ ๋ƒˆ๋Š”๋ฐ, RL๋กœ ํ•™์Šต๋œ Policy์— ๋”ฐ๋ผ ๋ฐ”๋กœ NN์„ ์ตœ์ ํ™”ํ•œ๋‹ค.
  • ์ด๊ฑฐ DevDay์— ๊ด€๋ จ ์„ธ์…˜์žˆ์—ˆ๋˜ ๊ฒƒ ๊ฐ™์€๋ฐ..
  • ์•„๋งˆ ์ดํ•ด๊ฐ€ ์ž˜ ์•ˆ๊ฐˆ ๊ฒƒ ๊ฐ™์•„์„œ ์จ๋‘๋ฉด, ์—ฐ์‚ฐ ์ตœ์ ํ™”๋‹ค.
  • https://arxiv.org/abs/2011.14486 ๋‚˜์ค‘์— ์ฝ์–ด๋ณด๋ฉด ์žฌ๋ฐŒ๊ฒ ๋‹ค ใ…‹ใ…‹
  • AutoTVM๋ณด๋‹ค ์›”๋“ฑํ•œ ์„ฑ๋Šฅ

The Hugging Face Ecosystem

  • ์ด๊ฑฐ breakout session์œผ๋กœ ๋“ค์–ด์„œ ํŒจ์Šค!

Performance & Profiler

Introducing New PyTorch Profiler

  • ์š”๊ฑฐ ์˜คํ”„๋‹ ํ†ก์œผ๋กœ ๋“ค์–ด์„œ ํŒจ์Šค!

TRTorch: A Compiler for TorchScript Targeting NVIDIA GPUs with TensorRT

  • NVIDIA
  • TensorRT + TorchScript
  • trtorchc ๋ผ๋Š” ๋ช…๋ น์–ด๋ฅผ ์จ๋†“์•˜๊ธธ๋ž˜ ๋ญ์ง€..? ํ–ˆ๋”๋‹ˆ TensorRt TORCH Compiler๋ฅผ ์ค„์ธ๋“ฏ
  • jit์œผ๋กœ trace๋œ ๋ชจ๋ธ ํŒŒ์ผ์„ compileํ•˜๋ฉด ๋จ
  • A100์—์„œ TensorRT๋ฅผ ์‚ฌ์šฉํ–ˆ์„ ๋•Œ object detection model(https://pytorch.org/hub/nvidia_deeplearningexamples_ssd/์š”๊ฑฐ์ธ๋“ฏ) ๊ธฐ์ค€์œผ๋กœ FP32 JIT ๋Œ€๋น„ FP16์„ฑ๋Šฅ์ด 14x ๋†’์Œ
  • Post Training Quantization๋„ ์จ๋†“์•˜๋Š”๋ฐ, ์ด๊ฑด ๋ญ ์˜ˆ์ „์—๋„ ์ž˜ ๋˜์—ˆ์—ˆ๋˜ ๊ฒƒ ๊ฐ™๋‹ค.
  • ๊ทผ๋ฐ ํ•œ 1๋…„ ๋ฐ˜์ฏค ์ „์— TensorRT๋ฅผ ์“ฐ๋ ค๊ณ  ํ—€๋‹ค๊ฐ€ Variable Length์—์„œ ์ž˜ ๋™์ž‘ํ•˜์ง€ ์•Š์•„์„œ ํž˜๋“ค์—ˆ๋˜ ๊ธฐ์–ต์ด ์žˆ๋‹ค. ์ž˜ ๋˜๋ฉด ์„ฑ๋Šฅ์ด ๋ฌด์กฐ๊ฑด ์ข‹์•„์ง€๊ธฐ ๋•Œ๋ฌธ์— ์จ๋ณด๊ณ  ์‹ถ๊ธด ํ•œ๋ฐ, ์ง€๊ธˆ ์–ด๋–จ์ง€๋Š” ํ™•์ธํ•ด๋ด์•ผ๊ฒ ๋‹ค.

Platforms & Ops & Tools

FairTorch: Aspiring to Mitigate the Unfairness of Machine Learning Models

  • ์ด๊ฑฐ PyTorch Global Hackathon์—์„œ ๋ณธ ๊ฒƒ ๊ฐ™์€๋ฐ..? ์ด๋Ÿฐ ํ”„๋กœ์ ํŠธ ์ด์–ด๊ฐ€๋Š” ์‚ฌ๋žŒ๋“ค๋„ ๋Œ€๋‹จํ•˜๊ณ  ๊ณ„์†ํ•ด์„œ ์ด๋Ÿฐ ์‚ฌ๋žŒ๋“ค์„ ์ปค๋ฎค๋‹ˆํ‹ฐ์— ๋Œ๊ณ ์˜ค๋Š” PyTorch๋„ ๋Œ€๋‹จํ•˜๋‹ค.
  • https://github.com/wbawakate/fairtorch
  • ๊ต‰์žฅํžˆ ๋ฏผ๊ฐํ•œ ํ”ผ์ณ๋“ค(race, gender) ๋“ฑ๋“ฑ์„ ๋“ค์–ด๊ฐ„ ์ถ”๋ก  ๊ฒฐ๊ณผ์—์„œ ์ตœ๋Œ€ํ•œ ํ†ต๊ณ„์  ์ฐจ์ด๋ฅผ ์—†์• ๋ ค๊ณ  ํ•œ๋‹ค.
  • ์“ธ ๊ฒƒ ๊ฐ™๊ธด ์•Š์•„๋„ ๊ต‰์žฅํžˆ ์žฌ๋ฐŒ๋Š” ํ”„๋กœ์ ํŠธ ๊ฐ™์ด์„œ ์ •๋ฆฌ

Vision

PyTorchVideo: A Deep Learning Library for Video Understanding

  • https://pytorchvideo.org
  • FAIR ๋‚ด๋ถ€์—์„œ ๋งŒ๋“ ๊ฑด๊ฐ€๋ณด๋‹ค
  • ์ผ๋‹จ PyTorch ๊ตฌํ˜„์— ๋น„ํ•ด ๋น ๋ฅด๋‹ค๋Š”๋ฐ, ๋‚ด๋ถ€ ์ตœ์ ํ™”๋ฅผ ์ข€ ์ž˜ ํ•ด๋†“์€ ๊ฒƒ ๊ฐ™๋‹ค. <- ๋‚˜์ค‘์— ์‚ดํŽด๋ด์•ผ์ง€
  • ๋น„๋””์˜ค ์ชฝ๋„ ๋ฐฐ๊ฒฝ์ง€์‹์ฒ˜๋Ÿผ ์•Œ์•„๋‘๋ฉด ์œ ์šฉํ•  ๊ฒƒ ๊ฐ™์•„์„œ ๊ทธ๋ƒฅ ๋‚ด์šฉ๋งŒ ์ฝ์–ด๋ดค๋‹ค.

PyTorch 3D: Fast, Flexible 3D Deep Learning

  • Facebook AI์—์„œ ๊ฐœ๋ฐœ
  • https://pytorch3d.org
  • https://arxiv.org/abs/2007.08501 ๊ด€๋ จ ๋…ผ๋ฌธ์ธ๋ฐ ์ฝ์–ด๋ณด๋ฉด ์ข‹์„ ๋“ฏ
  • ํšŒ์‚ฌ์—์„œ ์“ฐ์ž„์ƒˆ๋ฅผ ์ฐพ๊ธด ์–ด๋ ต๊ณ , ๋‹น์žฅ ์“ธ ๊ฒƒ ๊ฐ™์ง€๋„ ์•Š์ง€๋งŒ, ์–ธ์  ๊ฐ€ 3D ๋ฐ์ดํ„ฐ ๋‹ค๋ค„๋ณด๊ณ  ์‹ถ์–ด์„œ ์ฝ์–ด๋ดค๋‹ค.

CompressAI: a Research Library & Evaluation Platform for End-to-End Compression

  • https://github.com/InterDigitalInc/CompressAI
  • AutoEncoder๋กœ encoding(compress), decoding(decompress)ํ•˜๋Š” ๊ฒƒ์„ ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌํ™” ํ–ˆ๋‚˜๋ณด๋‹ค.
  • TF ์œ ์ €๋ฉด https://github.com/tensorflow/compression ์ฐธ๊ณ ํ•˜๋ฉด ๋  ๋“ฏ
  • End-to-End๋กœ jpeg๋ณด๋‹ค N๋ฐฐ์ž‘์€ ์ด๋ฏธ์ง€๋ฅผ ์ฃผ๊ณ  ๋ฐ›์„ ์ˆ˜ ์žˆ๋‹ค๋ฉด ๋ชจ๋ฐ”์ผ์—์„œ ์จ๋ณผ๋งŒ ํ•˜์ง€ ์•Š์„๊นŒ..?
  • ์ด๊ฑฐ ์—ญ์‹œ ๋‚˜์ค‘์— ๋ณผ ์šฉ๋„๋กœ ๋ฉ”๋ชจ

Breakout Session

Huggingface Ecosystem

  • ์ฒซ๋ฒˆ์งธ ์‹œ๊ฐ„์—์„œ ์ œ์ผ ์žฌ๋ฐŒ์–ด๋ณด์˜€๋‹ค.
  • ๋ชจ๋ธ ํ—ˆ๋ธŒ 2.0
  • HuggingFace Datasets
    • ์ด๊ฑฐ ์ข‹๊ธดํ•œ๋ฐ, ์ฐจ๋ผ๋ฆฌ PyTorch๋งŒ ํƒ€๊ฒŸํ•˜๊ณ  ๋ง˜๋จน๊ณ  ์ง€์›ํ–ˆ์œผ๋ฉด ๋” ์ข‹์•˜๊ฒ ๋‹ค๋ผ๋Š” ๋Š๋‚Œ
    • https://huggingface.co/datasets?filter=languages%3Ako
  • huggingface_hub ๋ผ๋Š” ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ ์ฐธ๊ณ ํ•ด๋ณด์ž
  • HuggingFace Ecosystem
    • Hub์—์„œ ๋ถˆ๋Ÿฌ์™€์„œ Transformers + Datastes + Tokenizers๋กœ ํ•™์Šต์‹œํ‚ค๊ณ , Model hub๋กœ ๋‹ค์‹œ ์˜ฌ๋ ค์„œ Inference API๋กœ ์‚ฌ์šฉํ•˜๊ฒŒ ํ•œ๋‹ค.
    • ๋ฐ์ดํ„ฐ์…‹๋งŒ ์ž˜ ์žˆ์œผ๋ฉด ๋‹ค ํ†ตํ•ฉ๊ฐ€๋Šฅํ•˜๊ธด ํ•  ๋“ฏ.
    • AWS EC2 Inf1๋„ ์ง€์›..?
  • ๊ฑฑ์ •๋˜๋Š” ์ 
    • ํ”„๋กœ๋•์…˜ ํ™˜๊ฒฝ์—์„œ ์กฐ๊ธˆ ํŠน์ดํ•œ ๋ฐ์ดํ„ฐ์…‹ ๋„ฃ์„ ๋•Œ ์ด์ œ ๋์—†๋Š” ๊ฐœ์กฐ๊ฐ€ ๋˜์ง€ ์•Š์„๊นŒ..?
    • -> ํ™•์‹คํžˆ ์ง€๊ธˆ๊นŒ์ง€๋Š” ๋‹ค์‹œ ์งœ๋Š”๊ฒŒ ๋น ๋ฅด๋‹ค๊ณ  ์ƒ๊ฐํ–ˆ๊ณ  ๊ทธ๋žฌ๋Š”๋ฐ, ์ด์ œ๋Š” ํ™•์ธํ•ด๋ด์•ผ ํ•  ๊ฒƒ ๊ฐ™๋‹ค.
  • accelerate๊ฐ€ ๋‚˜์˜ค๋Š”๋ฐ ์กฐ๊ธˆ์€.. ๋ถ€์ •์ ..
    • ์ด๊ฑธ ์™œ torch ์ชฝ์—์„œ ์ฒ˜๋ฆฌํ•˜์ง€ ์•Š๊ณ  ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ ์ชฝ์—์„œ ์ฒ˜๋ฆฌํ•ด์•ผํ•˜์ง€??๋ผ๋Š” ์ƒ๊ฐ์ด ๋“œ๋Š”๋ฐ, 1~2๋…„์‚ฌ์ด์— ํŒŒ์ดํ† ์น˜ ๋‚ด๋ถ€์—์„œ ๋ณ€ํ™”๊ฐ€ ์ƒ๊ธฐ๊ฒŒ ๋˜๋ฉด ๋์—†๋Š” ๋ ˆ๊ฑฐ์‹œ + ์˜์กด์„ฑ ๋ฒ„์ „์˜ ์ง€์˜ฅ์ด ์‹œ์ž‘๋˜์ง€ ์•Š์„๊นŒ? -> ๋ฌผ๋ก  ์‹คํ—˜ ์ฝ”๋“œ๋Š” ์ด๋Ÿฐ๊ฑฐ ์ ๊ทน์ ์œผ๋กœ ์จ๋„ ์ƒ๊ด€์—†๋‹ค๊ณ  ์ƒ๊ฐํ•œ๋‹ค.
    • DeepSpeed, FairScale ๊ธ‰์ด ์•„๋‹ˆ๋ฉด ์•ˆ์จ๋„ ๊ดœ์ฐฎ์ง€ ์•Š์„๊นŒ

Constrained Optimization in PyTorch 1.9 Through Parametrizations

https://pytorch.org/docs/master/generated/torch.nn.utils.parametrize.register_parametrization.html

์ œ๋ชฉ์ด ์‹ ๊ธฐํ•ด์„œ ๋“ค์–ด๊ฐ€๋ณด์•˜๋‹ค. ์œ„ ๋ชจ๋“ˆ ์„ค๋ช…์ธ๋ฐ, weight์— constraint๋ฅผ ๊ฑธ์–ด์„œ optimizeํ•˜๋Š” ๋‚ด์šฉ์ด๊ณ , ํŠน์ • ๋ถ„์•ผ์—์„œ ์“ธ๋งŒํ•œ ๋‚ด์šฉ๊ฐ™์•„ ๋ณด์ธ๋‹ค. ์„ธ์…˜์—์„œ ๋‚˜์˜จ ์˜ˆ์‹œ๋Š” symmetricํ•œ weight๋ฅผ ์‚ฌ์šฉํ•˜๋„๋ก ์ž‘์„ฑํ–ˆ๋‹ค.

์•„๋ž˜๊ฐ™์€ ๋ฐฉ์‹์œผ๋กœ ์‚ฌ์šฉ๊ฐ€๋Šฅ.

import torch
from torch.nn.utils import parametrize

class Symmetric(torch.nn.Module):
    def forward(self, X):
        return X.triu() + X.triu(1).T

linear = torch.nn.Linear(5, 5)
parametrize.register_parametrization(m, "weight", Symmetric())

assert torch.allclose(linear.weight, linear.weight.T)

๊ตฌํ˜„์€ property๋กœ ๊ตฌํ˜„ํ•œ ๋“ฏ ํ•˜๋‹ค. type์€ ํ…์„œ๊ฐ€ ์•„๋‹ˆ๋ผ property object๊ฐ€ ๋‚˜์˜จ๋‹ค. ์›๋ž˜ weight์€ .original suffix๊ฐ€ ๋ถ™์€์ฑ„๋กœ ๋‹ค๋ฅธ ๊ณณ์— ํ• ๋‹น๋œ๋‹ค.

caching์€ contextmanager๋ฅผ ํ†ตํ•ด ์ˆ˜ํ–‰ํ•˜๋Š”๊ตฌ๋‚˜.

์ˆ˜ํ•™์ ์ธ ๋ฒ ์ด์Šค๊ฐ€ ๊ฐ•ํ•œ ์‚ฌ๋žŒ์ด ๋‹ค์–‘ํ•œ ์ชฝ์œผ๋กœ ์‚ฌ์šฉ๊ฐ€๋Šฅํ•ด๋ณด์ด๋Š”๋ฐ, ๋‚˜๋Š” ์–ด๋–ป๊ฒŒ ํ™œ์šฉํ• ์ง€ ๋ชจ๋ฅด๊ฒ ๋‹ค.

Avalanche: an End-to-End Library for Continual Learning based on PyTorch

์ด๋Ÿฐ๊ฒŒ ์žˆ๊ตฌ๋‚˜.. ๋‚˜์ค‘์— ์‚ดํŽด๋ด์•ผ์ง€ https://avalanche.continualai.org

ํ›„๊ธฐ

๋‹ค์Œ๋‚  ํšŒ์‚ฌ๋Š” ํœด๊ฐ€๋ฅผ ์จ์„œ ๋„ˆ๋ฌด ํ”ผ๊ณคํ•ด์„œ ๋ชป ๋“ค์„ ๋•Œ๊นŒ์ง€ ๋ง˜ํŽธํ•˜๊ฒŒ ๋“ค์—ˆ๋‹ค. ๊ทธ๋‚˜์ €๋‚˜ ์ด๋ฒˆ์—๋„ ์–ด์ฉŒ๋‹ค๊ฐ€ ์ดˆ๋Œ€๋ฐ›์•˜๋Š”๋ฐ, ์‚ฌ์‹ค ๋‚ด๊ฐ€ ๋ญ˜๋กœ ์ดˆ๋Œ€๋ฐ›์€ ๊ฑด์ง€ ๊ถ๊ธˆํ•˜๋‹ค. ์ฃผ์œ„์— ์ฐธ์—ฌํ•˜๋Š” ์‚ฌ๋žŒ์ด ๊ฑฐ์˜ ์—†๊ธฐ๋„ ํ•˜๊ณ .

๊ทธ๋ž˜๋„ ์˜†์— Piotr Bialecki ๊ฐ™์€ ๋ถ„์ด ์žˆ๊ณ , ์ € ์˜†์— Thomas Wolf๊ฐ™์€ ๋ถ„์ด ํ–‰์‚ฌ ์ฐธ๊ฐ€ํ•˜๊ณ  ์žˆ๋Š” ๊ฒƒ์„ ๋ณธ๋‹ค๋Š” ๊ฒŒ ๋‚˜์—๊ฒŒ๋Š” ์ข‹์€ ๊ฒฝํ—˜์ด๊ธฐ ๋–„๋ฌธ์— ์ดˆ๋Œ€ ๋ฐ›์€ ๊ฒƒ์ด ๋งŽ์ด ๊ฐ์‚ฌํ•˜๋‹ค. ์ถ”๊ฐ€๋กœ ๋„คํŠธ์›Œํ‚น ์„ธ์…˜์—์„œ ์–ด๋–ป๊ฒŒ PyTorch ๊ธฐ์—ฌ๋ฅผ ์‹œ์ž‘ํ–ˆ์—ˆ๋Š”์ง€ ์ฐ ํ‘ธ๋Š” ๊ฒƒ๋„ ๋“ฃ๊ณ , ์กฐ์–ธ๋„ ๋“ฃ๊ณ  ํ–ˆ์œผ๋‹ˆ..

์ค‘๊ฐ„์ค‘๊ฐ„์— ์—ฌ๋Ÿฌ ํšŒ์‚ฌ์—์„œ ๋‚˜์˜จ ์‚ฌ๋žŒ๋“ค๊ณผ ์ด์•ผ๊ธฐํ•ด๋ณด๋Š”๋ฐ, ์–ด์ฉŒ๋‹ค๊ฐ€ ํ•œ ์ƒˆ๋ฒฝ ์„ธ์‹œ์ฏค์— ํšŒ์‚ฌ๊ฐ€ ์–ด๋”˜์ง€ ์–˜๊ธฐ๊ฐ€ ๋‚˜์™€์„œ ํ•œ๊ตญ์— ์žˆ๋Š” ํšŒ์‚ฌ๋ผ๊ณ  ํ–ˆ๊ณ  ๋ช‡์‹œ์ธ์ง€ ๋ฌป๊ธธ๋ž˜ ๋Œ€๋‹ตํ–ˆ๋”๋‹ˆ ๋งŽ์ด ๋†€๋ผ๋Š” ๊ฒŒ ์›ƒ๊ฒผ๋‹ค. ์˜จ๋ผ์ธ ์ปจํผ๋Ÿฐ์Šค๋Š” ๋‹น์—ฐํžˆ ์ƒˆ๋ฒฝ์ด๋ผ ์ƒ๊ฐํ–ˆ๋Š”๋ฐ, ์ƒ๊ฐํ•ด๋ณด๋‹ˆ ์ฃผ์ตœ์ธก์—์„œ๋Š” ๋‚ฎ์‹œ๊ฐ„์ด๊ฒ ๊ฑฐ๋‹ˆ ์‹ถ์–ด์„œ ๋‚ฉ๋“์ด ๋˜๊ธฐ๋„ ํ•˜๊ณ . ๊ทธ๋ ‡๊ฒŒ ์Šค๋ชฐํ†ก๋„ ๋‚˜๋ˆ„๊ณ  ์—ฌ๋Ÿฌ ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ ์‚ฌ์šฉ ์ผ€์ด์Šค ์„ค๋ช…๋“ฃ๊ณ , ํšŒ์‚ฌ์—์„œ ์–ด๋–ป๊ฒŒ ์“ฐ๋Š”์ง€ ์ผ๋Š”์ง€ ์ด์•ผ๊ธฐ ๋‚˜๋ˆ„๋Š”๋ฐ ์ •๋ง ์‹ ๊ธฐํ–ˆ๋‹ค.

์ค‘๊ฐ„์— ์ ‘์†์ด ์•ˆ๋˜์–ด์„œ ์—„์ฒญ ๋‹นํ™ฉํ–ˆ๋Š”๋ฐ, Gather Town ์žฅ์• ์ด๊ธด ํ•˜์ง€๋งŒ, ์ด๋ ‡๊ฒŒ ํฐ ํ–‰์‚ฌ๋„ ๋‹ค์šด๋˜๊ธฐ๋„ ํ•˜๋Š”๊ตฌ๋‚˜ ์‹ถ์—ˆ๋‹ค. ๊ทธ๋ฆฌ๊ณ  PyTorch Dev Day 2020๋•Œ๋ณด๋‹ค ๋„คํŠธ์›Œํ‚น ์„ธ์…˜๋„ ๋” ์ž˜ ๋˜์–ด์žˆ๋Š” ๊ฒƒ์„ ๋ณด๋‹ˆ ์ง„์งœ ํ–‰์‚ฌ ์ค€๋น„ํ•œ ๋ถ„๋“ค ๋Œ€๋‹จํ•˜๋‹ค.. ๐Ÿ‘

๊ทธ๋ฆฌ๊ณ  ์ค‘๊ฐ„์— ์•„๋Š” ์‚ฌ๋žŒ๋„ ๋งŒ๋‚ฌ๋‹ค
April 22, 2021
Tags: conference pytorch