The recent study of Chae et al. (2017) found a one-to-one correspondence between plasma blobs outflowing along a ray formed after a coronal mass ejection (CME) and small X-ray flares. In the present work, we have examined the spatial configuration and the eruption process of the flares that are associated with the blobs by analyzing EUV images and magnetograms taken by the SDO/AIA and HMI. We found that the main flare and the successive small flares took place in a quadrupolar magnetic configuration characterized by predominant magnetic fields of positive polarity, two minor magnetic fragments of negative polarity, and a curved polarity inversion line between them, which suggests that the formation process of the blobs may be similar to that of the parent CME. We also found that the successive flares resulted in a gradual change of the quadrupolar magnetic configuration, and the relevant migration of flaring kernels. The three-dimensional geometry and the property of the current sheet, that is often supposed to be embedded in an observed post-CME ray, seem to keep changing because of mutual feedback between the successive flares and the temporal change of the magnetic field configuration. Our results suggest that the observed post-CME rays may not reflect the characteristics of the current sheet responsible for the impulsive phase of the flare.