Erm, well technically the term "Black Hole" describes a postulate of breaking the strong nuclear force with gravitation, but this is indeed important only among theoretical physicists. It is especially important among alternative theorists because such a thing gives ground to Superstrings and m-Brane theory.
You are quite correct the term is often largely interchangeable among many astrophysicists. Most cosmologists find the distinction irrelevent. For all intents and purposes a Black Hole is a neutron star, however it is one for which its tidal mass (ie. gravitational force) is so great the math defies the strong nuclear force, hence neutrons could not exist.