A method for visual media processing includes processing a bitstream representation for a conversion between a current video block in a video region of a visual media data and the bitstream representation. The bitstream representation is based on a rule that specifies that one or more syntax elements used in a luma-dependent chroma residual scaling step are selectively included in the bitstream representation. The luma-dependent chroma residual scaling step includes scaling chroma samples based on neighboring reconstructed luma samples during the conversion. The one or more syntax elements are usable to derive a chroma residual scaling factor used for the scaling.