In general, it is advised that you play the role you're given, though in your case I wouldn't blame you if you didn't want to do that. City Hall, the police station, and the dairy are all off limits. Every so often the town will also have strange things happening that in general cause everyone to be much more hostile than usual.

I've heard people have been able to get home and return, but it's probably better to ask around and get the story from those people in person.

Oh, and rumor has it that it's not a good idea to drink the milk.