A method is used in managing inline data compression in storage systems. A request is received to flush data of a file system to a storage device of a storage system. A compression ratio is determined for writing the data to the storage device in a compressed format. Based on the determination, a number of allocation units is determined in which a segment is apportioned. The segment comprises of a set of allocation units in the compressed format. Based on the determination, the data is written to the segment in the compressed format.