For guidance on suitable antennas for digital TV reception, you might try the website "antennaweb.org". The site is intended for home reception, and so the questions anticipate a fixed location. If there are a few known locations where you would like to deceive DTV in a boat, try describing them per the questions on antennaweb. Perhaps a suitable antenna will emerge. If the locations are distant from transmitters or in locations that showed strong ghosts with analog TV, then the recommended antenna might not be practical for boats.
Supposedly, newer antennas are being brought to market that steer their reception patterns electronically (no moving parts) and automatically to optimize gain and minimize interference. Both the antennas and the TV to which they are connected must have a "CEA-909" interface to make the automatic features work. I'm not sure of the availability of these antennas at retail.
In general, DTV operates well with much poorer signal conditions than analog TV, but individual stations may vary because of different transmitter location, different transmitter antenna pattern, some peculiar "ghost" (multipath) conditions, etc. In the case described in this thread, it seems likely that the new receiver antenna was quite different (was the new antenna connected properly?); if so, attempts at comparisons are probably not valid.
-- John H