Really? You're just going to walk right in there?

[I mean, he's not moving to stop him. Though he looks like he might try any second now.]

You do realize that's clearly what they want you to do, right? Hey, welcome to my school prison for magical children. I'm clearly powerful enough to bring you all here against your will, but I can't stop you from going in the Dining Hall! Please don't, though! It's the one place you can't go! I'm asking nicely!

Bullshit. They're goading you.