Understanding Transformer-Based Self-Supervised Architectures — With the burgeoning of Transfer Learning, Deep Learning has achieved many wonders. More specifically, in NLP, with the rise of the Transformer (Vaswani et. al.), …

Understanding Transformer-Based Self-Supervised Architectures — Transformer (Vaswani et. al.) is great, it attends to a longer context, it offers parallelization in computation which RNNs don’t, and most importantly, they have the state of the art results. In this article, we’ll be covering the Reformer Model, which was proposed in the paper Reformer: The Efficient Transformer…

Training Faster RNNs with Quasi-RNN — Recurrent Neural Networks (RNNs) have been in the sequence modeling business for a long time. But RNNs are slow; they process one token at a time. Moreover, the recurrent architecture adds a limitation of fixed-length encoding vectors for the complete sequence. …

Generating Meaningful Data from Noise — This (GANs), and the variations that are now being proposed is the most interesting idea in the last 10 years in ML, in my opinion — Yann LeCun (VP & Chief AI Scientist at Facebook) in his Quora Session GANs are one of the latest wonders of Deep Learning. These…

How I did at the Google Foobar Challenge, Twice! — Foobar is “said to be” Google’s secret hiring challenge, and it’s really exciting. It is an invite-only challenge, so “you have to be chosen.” The picture above is how I received my invite. In this article, I’ll be sharing my experience of the foobar challenge. …

Understanding Transformer-Based Self-Supervised Architectures — In this article, we’ll be discussing the renowned GPT-3 model proposed in the paper “Language Models are Few-Shot Learners” by OpenAI. It is the successor of GPT-2, which has a very similar architecture to that of GPT-3. If you’re unaware of GPT-2, consider giving my article on GPT-2 a read…

Towards Training Better Reinforcement Learning Agents — Dynamic Programming is a mathematical optimization approach typically used to improvise recursive algorithms. It basically involves simplifying a large problem into smaller sub-problems. There are two properties that a problem must exhibit to be solved using dynamic programming: Overlapping Subproblems Optimal Substructure We’ll be discussing ‘Planning in RL’ using dynamic…

Understanding Transformer-Based Self-Supervised Architectures — Transformer-based language models have been leading the NLP benchmarks lately. Models like BERT, RoBERTa have been state-of-the-art for a while. However, one major drawback of these models is that they cannot “attend” to longer sequences. For example, BERT is limited to a max of 512 tokens at a time. To…

Optimizing GPU Performance with TensorFlow — We want our models to train real fast. We use GPUs to make operations execute faster. However, it is possible that even after speeding up the computations, the model may have inefficiencies in the pipeline itself, and thus, may train slower. …