you can use "Reflection file editor" for this. It is available from PHENIX GUI. I'm not sure if the corresponding command line option exists, Nat knows this better.
In a hurry right now, but to summarize: iotbx.reflection_file_editor will do this on the command line, but it's intended as a tool for automation and testing, and is currently undocumented. Something like this should do it:
iotbx.reflection_file_editor data.mtz r_free_flags.extend=True
but I'd really recommend trying the GUI first. FYI, it assumes that R-free flags were initially generated randomly, not by thin resolution shells (i.e. using DATAMAN or SFTOOLS).
-Nat