Teaching a tool to draw
I wanted drawings for this site that were mine. Not a style borrowed from a model that has seen ten million pictures, and not a filter.
What's the tool
For me, it's beautiful, abstract, sensitive. But I can also see why this is a basic photo transformation tool. It reads a photograph, its pixels, its light, its tone and tries to find its essence and its movement, guided by what I know a drawing should be. What makes me love it is the imperfection of the result, just like most sketches are in reality.
The feedback process
It started shaky and unsure, the way Adobe Illustrator does when I ask it to vectorize a photo. It throws away the floor and walks the edge of what is left, the way you would with a pencil without lifting it. Then it re-spaces that walk so the pixel grid stops showing, smooths it while leaving the sharp turns exactly where they are, breaks it into separate strokes so the shape never closes, and draws each stroke with a pen that lands thin, swells, and lifts thin.
Nothing is generated. There is no model, no prompt and no token. Every line is measured off the photograph, and controlled with a squint feature, with a speed, movement of the lines, which is why it can be wrong in ways you can point at.
The process has value
The first version returned nine points and a straight line. Later it traced the stage instead of the dancer, because the brightest thing in a theatre photograph is the floor. Then we had a clean silhouette, but a clean silhouette reads as a cut-out rather than as a drawing. The fourth was smooth but still stepped, because it followed the photograph one pixel at a time and carried the staircase of the grid into the line.
Portraits were more emotional, and the fix was the opposite of what I assumed. The lines wobbled, so I made them "smoother", which translated into thinner. That made it worse. A thin line has nothing to absorb a wobble; a fuller line swallows it. I had the diagnosis backwards.
The instruction that changed it
The best direction I gave was not technical: Continue the lines further than you can see them, and let them fade in.
That is how I draw. You do not stop where the pixel stops. You finish the shape you were making. You enjoy the push to the limit, the imagination that a muscle continues behind the skin, in the shadows, you predict the volume, the tension. Now each line is carried on along the curve it was already travelling, so a round shape keeps rounding, and a straight one is left alone because a straight line has nothing to finish.
No diffusion, no image model, no generation. Every line is actually drawn from a photograph.
Why it is hard
An image generator gives you something plausible in one second. Teaching a tool to draw means watching it fail in a way you can name, and fixing the cause rather than the picture.
What changed
You can't draw what you can't see, and drawing suggests more than defines.
Working with AI
An image generator would have used diffusion. Based on its training it would have looked acceptable in one second, I would have learned nothing, and it would not have been mine.
Instead I made it mine, step by step, not by saying "make it better", but by pointing at the drawing itself: focus less on the contrast, let go of the continuous lines, follow where the contrast fades, and start again. It stopped at corners because a corner is exactly where the next pixel is not touching the last one.
You cannot give an agent that instruction unless you have looked hard enough to know what is actually wrong. That is the work and it is fascinating. We are working on improving the sensitivity of his stroke next.