If you have ever tried to remove the background from a photo of a wine bottle, a glass of water, or a window, you have seen the problem. The cutout either keeps the background inside the object or erases the object entirely. Neither is what you wanted.
Every segmentation model makes a simple assumption: a pixel belongs to the foreground or the background. For opaque objects, this is true. For transparent objects, it is false.
A pixel in the middle of a glass of water is not foreground or background. It is both. It shows the colour of the water mixed with whatever is behind the glass. If you remove the background, you also remove the colour that was showing through the water.
The same applies to glass, windows, smoke, steam, and any material that is partially see-through. The model has no way to separate the material from the background behind it because they are mixed together in the pixel values.
Most models will treat the transparent object as background if the colour is close to the background colour. This produces an invisible object or a partial silhouette.
Some models will treat the entire transparent object as foreground, keeping the background colour inside it. When you composite over a new background, the old background shows through the glass.
The best models, like BiRefNet, will attempt to estimate a partial alpha for the glass edges while filling the interior with a guess. The result is inconsistent: the edges are handled, the interior is not.
For transparent objects, the workflow that actually works involves compositing rather than removal. You keep the object against its original background, or you shoot it against a background you want to keep.
If you need the object on a different background, you shoot it twice. Once against a bright background and once against a dark one. Then you use a matting algorithm that compares the two images to estimate the transparency of each pixel. This is called difference matting.
Difference matting produces excellent results for glass and transparent objects. It requires control over the shooting environment, which means it is not a browser-tool workflow. It is a studio workflow.
If you are selling glass or transparent products, the standard ecommerce practice is to photograph them against white and leave the background in place. The product appears on a white background, which is what marketplaces want.
Removing the background from a glass bottle produces a cutout that looks broken when placed on a non-white background. The white background is not a compromise, it is the correct choice.
If you need a lifestyle image with the product in a scene, use a mockup or a composite that keeps the original background but places it in a larger layout. This avoids the transparency problem entirely.
No background removal tool handles glass correctly. This is a fundamental limitation of segmentation models, not a bug. The solution is to shoot against a controlled background or to use difference matting.
You can improve it in a photo editor by reconstructing the transparency by hand. This is time-consuming and produces inconsistent results. The shooting approach is faster.
Same problem, worse. Smoke has no edges at all, only a gradual transition from dense to transparent. No current model handles it automatically.
Shadows are a partial case. A hard shadow can be removed as background. A soft shadow is a gradient and the model will usually remove it entirely or keep it entirely. Neither is ideal.