Curated list of awesome GAN applications and demo

gans-awesome-applications

Curated list of awesome GAN applications and demonstrations.

Note: General GAN papers targeting simple image generation such as DCGAN, BEGAN etc. are not included in the list. I mainly care about applications.

The landmark papers that I respect.

• Generative Adversarial Networks, [paper], [github]
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks, [paper], [github]
• Improved Techniques for Training GANs, [paper], [github]
• BEGAN: Boundary Equilibrium Generative Adversarial Networks, [paper], [github]

Contents

Use this contents list or simply press command + F to search for a keyword

• Applications using GANs
  • Font generation
  • Anime character generation
  • Interactive Image generation
  • Text2Image (text to image)
  • 3D Object generation
  • Image Editing
  • Face Aging
  • Human Pose Estimation
  • Domain-transfer (e.g. style-transfer, pix2pix, sketch2image)
  • Image Inpainting (hole filling)
  • Super-resolution
  • High-resolution image generation (large-scale image)
  • Adversarial Examples (Defense vs Attack)
  • Visual Saliency Prediction (attention prediction)
  • Object Detection/Recognition
  • Robotics
  • Video (generation/prediction)
  • Synthetic Data Generation
  • Others
• Did not use GAN, but still interesting applications
  • Real-time face reconstruction
  • Super-resolution
  • Photorealistic Image generation (e.g. pix2pix, sketch2image)
  • Human Pose Estimation
  • 3D Object generation
• GAN tutorials with easy and simple example code for starters
• Implementations of various types of GANs collection
• Trendy AI-application Articles

Applications using GANs

Font generation

• Learning Chinese Character style with conditional GAN, [blog], [github]

Anime character generation

• Towards the Automatic Anime Characters Creation with Generative Adversarial Networks, [paper]
• [Project] A simple PyTorch Implementation of Generative Adversarial Networks, focusing on anime face drawing, [github]
• [Project] A simple, clean TensorFlow implementation of Generative Adversarial Networks with a focus on modeling illustrations, [github]
• [Project] Keras-GAN-Animeface-Character, [github]
• [Project] A DCGAN to generate anime faces using custom mined dataset, [github]

Interactive Image generation

• Generative Visual Manipulation on the Natural Image Manifold, [paper], [github]
• Neural Photo Editing with Introspective Adversarial Networks, [paper], [github]

Text2Image (text to image)

• TAC-GAN – Text Conditioned Auxiliary Classifier Generative Adversarial Network, [paper], [github]
• StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks, [paper], [github]
• Generative Adversarial Text to Image Synthesis, [paper], [github], [github]
• Learning What and Where to Draw, [paper], [github]

3D Object generation

• Parametric 3D Exploration with Stacked Adversarial Networks, [github], [youtube]
• Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling, [paper], [github], [youtube]
• 3D Shape Induction from 2D Views of Multiple Objects, [paper]
• Fully Convolutional Refined Auto-Encoding Generative Adversarial Networks for 3D Multi Object Scenes, [github], [blog]

Image Editing

• Invertible Conditional GANs for image editing, [paper], [github]
• Image De-raining Using a Conditional Generative Adversarial Network, [paper], [github]

Face Aging

• Age Progression/Regression by Conditional Adversarial Autoencoder, [paper], [github]
• CAN: Creative Adversarial Networks Generating “Art” by Learning About Styles and Deviating from Style Norms, [paper]
• FACE AGING WITH CONDITIONAL GENERATIVE ADVERSARIAL NETWORKS, [paper]

Human Pose Estimation

• Pose Guided Person Image Generation, [paper]

Domain-transfer (e.g. style-transfer, pix2pix, sketch2image)

• Image-to-Image Translation with Conditional Adversarial Networks, [paper], [github], [youtube]
• Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks, [paper], [github], [youtube]
• Learning to Discover Cross-Domain Relations with Generative Adversarial Networks, [paper], [github]
• Unsupervised Creation of Parameterized Avatars, [paper]
• UNSUPERVISED CROSS-DOMAIN IMAGE GENERATION, [paper]
• Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks, [paper], [github]
• Pixel-Level Domain Transfer [paper], [github]
• TextureGAN: Controlling Deep Image Synthesis with Texture Patches, [paper], [demo]
• Vincent AI Sketch Demo Draws In Throngs at GTC Europe, [blog], [youtube]
• Deep Photo Style Transfer, [paper], [github]

Image Inpainting (hole filling)

• Context Encoders: Feature Learning by Inpainting, [paper], [github]
• Semantic Image Inpainting with Perceptual and Contextual Losses, [paper], [github]
• SEMI-SUPERVISED LEARNING WITH CONTEXT-CONDITIONAL GENERATIVE ADVERSARIAL NETWORKS, [paper]
• Generative Face Completion, [paper], [github]

Super-resolution

• Image super-resolution through deep learning, [github]
• Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network, [paper], [github]
• High-Quality Face Image Super-Resolution Using Conditional Generative Adversarial Networks, [paper]
• Analyzing Perception-Distortion Tradeoff using Enhanced Perceptual Super-resolution Network, [paper], [github]

Image Blending

• GP-GAN: Towards Realistic High-Resolution Image Blending, [paper], [github]

High-resolution image generation (large-scale image)

• Generating Large Images from Latent Vectors, [blog], [github]
• PROGRESSIVE GROWING OF GANS FOR IMPROVED QUALITY, STABILITY, AND VARIATION, [paper], [github]

Adversarial Examples (Defense vs Attack)

• SafetyNet: Detecting and Rejecting Adversarial Examples Robustly, [paper]
• ADVERSARIAL EXAMPLES FOR GENERATIVE MODELS, [paper]
• Adversarial Examples Generation and Defense Based on Generative Adversarial Network, [paper]

Visual Saliency Prediction (attention prediction)

• SalGAN: Visual Saliency Prediction with Generative Adversarial Networks, [paper], [github]

Object Detection/Recognition

• Perceptual Generative Adversarial Networks for Small Object Detection, [paper]
• Adversarial Generation of Training Examples for Vehicle License Plate Recognition, [paper]

Robotics

• Unsupervised Pixel–Level Domain Adaptation with Generative Adversarial Networks, [paper], [github]

Video (generation/prediction)

• DEEP MULTI-SCALE VIDEO PREDICTION BEYOND MEAN SQUARE ERROR, [paper], [github]

Synthetic Data Generation

• Learning from Simulated and Unsupervised Images through Adversarial Training, [paper], [github]

Others

• (Physics) Learning Particle Physics by Example: Location-Aware Generative Adversarial Networks for Physics Synthesis, [paper], [github]
• (Games) STYLE TRANSFER GENERATIVE ADVERSARIAL NETWORKS: LEARNING TO PLAY CHESS DIFFERENTLY, [paper], [github]
• (General) Spectral Normalization for Generative Adversarial Networks, [paper], [github]

Did not use GAN, but still interesting applications.

Real-time face reconstruction

• Model-based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction, [paper], [github], [youtube]

Super-resolution

• Learning to Simplify: Fully Convolutional Networks for Rough Sketch Cleanup, [paper], [site link], [youtube]

Photorealistic Image generation (e.g. pix2pix, sketch2image)

• The Sketchy Database: Learning to Retrieve Badly Drawn Bunnies, [paper], [youtube]
• PatchMatch: A Randomized Correspondence Algorithm for Structural Image Editing, [paper], [github], [youtube]

Human Pose Estimation

• Knowledge-Guided Deep Fractal Neural Networks for Human Pose Estimation, [paper], [github]

3D Object generation

• 3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction, [paper], [github]

GAN tutorials with easy and simple example code for starters

• 1D Generative Adversarial Network Demo
• starter from “How to Train a GAN?” at NIPS2016
• NIPS 2016 Tutorial: Generative Adversarial Networks
• OpenAI - Generative Models
• [paper], [github], [youtube]

Implementations of various types of GANs collection

• nashory/gans-collections.torch, torch7
• hwalsuklee/tensorflow-generative-model-collections, pytorch
• wiseodd/generative-models, both pytorch and tensorflow
• aboev/arae-tf, tensorflow

Trendy AI-application Articles

• Artificial intelligence can say yes to the dress

Author

Minchul Shin, @nashory

Any recommendations to add to the list are welcome :)
Feel free to make pull requests!