New pins display the street address for that location by default. To create a custom name, tap the circled "i" on the right side of the pin's banner to open the settings panel. Tap in the name field to change the name and save.
To delete a pin, tap it on the map and a banner will appear with an X on the left side. Tapping the X will remove the pin from the map after a confirmation prompt.
Hint: To see the full list of your location pins, peel the map from the lower right corner and tap the Location List button.