WotC’s D&D Senior Manager Mike Mearls told Kotaku the step was taken to "make sure that the art we presented was as appealing to as wide an audience as possible." WotC did vigorous crowdsourced playtesting for D&D 5th Edition (see "New 'D&D' Dates, Info, and Pics"), and Mearls noted that "over 175,000 responses from early testers confirmed that gamers prefer elder brains and beholders, apparently, to monster boobs."
The game was aimed at a different demographic in the 1970s. The article cites a 1978 survey that found that the percentage of female D&D fans was a miniscule .4 to 2.3 percent of the total. And Mearls noted that in material that inspired D&D, including Greek myths and folk tales, as well in stories featuring D&D co-creator Gary Gygax favorite Conan the Barbarian, female characters were often depicted as femme fatales. “I imagine in the early, male-dominated years of the hobby that took root and became a cliche,” Mearls said.
Not all games are necessarily moving in the same direction. The portrayal of women in the world of Conan, and modern sensibilities regarding it, recently surfaced in criticism of Monolith’s Conan: The Board Game (distributed in North America by Asmodee NA, see "Asmodee Ex-Employee Attacks 'Conan: The Board Game,' Alleges Asmodee Threat").