`hasMask` in document inspection covers only raster masks, so a layer
with a vector mask looked unmasked to an agent. Layers, groups included,
now also report `hasVectorMask` and `vectorMaskEnabled` (null when there
is no vector mask). `hasMask` keeps its meaning.