"I miss my friends," Patrick J. Adams wrote on Instagram late Monday alongside a photo of Meghan Markle and Sarah Rafferty.