DoodleFormer: Creative Sketch Drawing

with Transformers

ECCV 2022

Ankan Kumar Bhunia1,    Salman Khan1,2,    Hisham Cholakkal1,    Rao Muhammad Anwer1,3
Fahad Shahbaz Khan1,4,    Jorma Laaksonen3,    Michael Felsberg4

2Australian National University, Australia
3Aalto University, Finland
4Linköping University, Sweden

Doodleformer imitates the way a human artist naturally draws sketches.

Generally, an artist first draws the holistic coarse structure of the sketch and then fills the fine-details to generate the final sketch. By first drawing the holistic coarse structure of the sketch aids to appropriately decide the location and the size of each sketch body part to be drawn.

We propose a novel two-stage encoder-decoder framework, DoodleFormer, for creative sketch generation. DoodleFormer decomposes the creative sketch generation problem into the construction of holistic coarse sketch composition followed by injecting fine- details to generate final sketch image.

  1. The first stage, PL-Net, takes the initial stroke points as the conditional input and learns to return the bounding boxes corresponding to each body part to be drawn;
  2. The second stage, PS-Net, takes the predicted box locations as inputs and generates the final sketch image;


  title={Doodleformer: Creative sketch drawing with transformers},
  author={Bhunia, Ankan Kumar and Khan, Salman and Cholakkal, Hisham and Anwer, Rao Muhammad and Khan, Fahad Shahbaz and Laaksonen, Jorma and Felsberg, Michael},