A method for visual media processing, including performing a conversion between a current chroma chroma block of visual media data and a bitstream representation of the current chroma chroma block, wherein, during the conversion, a chroma residual of the current chroma chroma block is scaled based on a scaling coefficient, wherein the scaling coefficient is derived at least based on luma samples located in predefined positions.