Isn't it generally used to mean the opposite of that? "Despite what I just said, I hold or will present the following apparently contrary position," more or less. Like if you spent a couple of paragraphs talking about the excellent cinematography of a film and then followed it with "That said, I didn't actually enjoy it. I found the protagonist insufferable."