Segmentation of freehand line drawings with neural networks

Abstract

Vector graphics have many advantages over raster graphics, including infinite scalability without loss of quality and being easier to manipulate programmatically. These properties make them useful in the field of art and animation, but most artists find it easier to work with raster graphics, and vectorization algorithms used by mainstream vector editors typically do not produce semantically correct vectorization results where stokes become separate curves that follow the stroke centreline and junctions have accurate connectivity. An algorithm for the segmentation of strokes in black and white freehand line drawings for the purpose of acting as a pre-processing step for vectorization is presented. A convolutional neural network is trained to detect junctions in a drawing. A second network is trained to predict which pixels belong to the same stroke. These networks are used in conjunction to find junction-adjacent pixels, and to make connectivity predictions about these pixels. Detected junctions are erased from the image and a graph is constructed from the unbranching stroke segments, which are combined based on the similarity of the connectivity predictions associated with the segments. The separate strokes of the original image are reconstructed based on the graph. These strokes are simple to vectorise as they have no branches or junctions. Both models were successful in making predictions about programmatically generated examples, but the junction model became very sensitive to the quality of stroke edges when trained with examples from real drawings. The results were promising but more development and a larger real-life dataset are needed before practical use.