Sides of The Fence

I'm on both sides of the fence, stuck between yes or no.
I don't want to get hurt but on the other side, I also don't want to let go.
Is that what people called selfish or so?