Office Hanky Panky

I’m really attracted to a guy I work with. I guess you could say we flirt all the time with sexual innuendos but nothing physical. I don’t want it to go down that road because we’re both married with kids. I really cherish my relationship with him and don’t want it to end. He has become a great friend and confidant. Is this really so bad if we don’t cross any lines? Liz