Messed up : I felt cornered and gave my number when a guy from a night club turned out to be my delivery man. Now I'm scared. Help Please!

Hello! Firstly, excuse my English please, I'm not a native speaker :) So a few weeks back I was at a club for a friend's Bday and talked with a guy while waiting for my friend. We kinda had a nice conversation about his job in the cinema industry etc. and I saw that he was acting interested but I kept my distance and said "bye!" when my friend was back. 3 weeks later, THAT guy shows up at my door and says my name and reminds me of that evening at the club, all smiles and "nice" vibes: "Sooo nice to see you again, we should get together, our convo was so nice!" He asked for my number and I SUPIDLY gave it to him because I felt already cornered as he now knows where I live. 2 weeks after (too long already I know), he proposed a shitty coffe and/or scooter tour date (nope!) last minute and I responded the following day, kindly (because I'm afraid) "Hello! Thanks for the invitation but I am not available this week-end, gladly some other time ☀️" He responded "With great pleasure 😊maybe in the following weeks if the weather is nice...have a nice sunday [my name spelled wrong again]" WHAT NOW? 😥 I know from this shitty/dangerous (#Scooter) date proposition that this is not the kind of guy I want to date but I felt 1. compelled to give my number not to antagonize him when put on the spot and 2. compelled to answer nicely and not block and delete because he knows my address. What kind of message can I write to let him down "nicely" (🤢) because I feel like my safety is at risk...? Thank you so much if one of you girls can help me 💕