Drift during integration causes uneven background

For some of my captures, star tracking fails occasionally (typically when the tracking star is too faint, or there is some wind) resulting in some overall drift over time. Most of my frames are still good, I just reject the ones where stars are elongated or have other artefacts.

Because of the drift, when I stack the result, I don't have the same number of stacked frames at every pixel.

Now I'm in a site with some light pollution, and so in positions in the final stacked image where I have many frames, the background looks very light and obvious. Were I have fewer frames, it is darker. The attached stacked image shows what I mean. The result is disappointing because it lack color and contrast.

This effect is very difficult to deal with on the stacked result. Is there a way to correct the background on every frame so it is darker, so when I combine all the frames, it stays dark everywhere ? In other words I would like to estimate the background light pollution pedestal on every frame and subtract it on each of these frames, so when I combine all the stacked images, it is not as bothering. Is there a way to do this in APP ?

On the attached image, see how the background is much darker near the top of the image than near the middle? this is not due to the C12 galaxy presence, but due to the light pollution background.

Thanks.