You can either make a footprint that uses the Edge.Cuts layer using the Footprint Editor, or draw the cutouts yourself in the Edge.Cuts layer in Pcbnew.
I made the cutout a footprint for reusability, but it is not natively supported in KiCAD's footprint editor and must be manually edited using a text editor. The example pic below shows what I did; the cutouts are footprints in this case.