Fannie, Freddie downgrades hurt banks, insurers