[ He's going to pointedly ignore that first statement and get to what matters. ]

I told them we've been meeting in secret for some time now.

[ Considering he didn't tell them, he considers it 'in secret'. ]

They didn't like that I was meeting you alone without telling anyone.