Meet in a crowded public place and it's sort of safe. In my area there have been criminals setting up meets to sell btc and then just mugging the buyer because they know they're carrying a bunch of cash to the meetup. Don't think you're totally safe because you are meeting at a Starbucks or something. They may agree to sell you a bitcoin at Starbucks and then stand you up only to mug you in the parking lot when you try to get in your car.

Just remember, there's a reason banks have all the security cameras, bullet proof glass for the teller windows, and security guards - they deal with cash.