So environmental laws are different across the country. I can speak for Washington and almost all development must have an extensive environmental review, but they are looking more at the biology aspect of the environment (flora / fauna). Permit's can't be issued until that review is done. The do look at a Phase 1 or 2, but those usually aren't an issue and any problems are addressed very early in the development process.
Most states however have requirements that if environmental or archeological issues are encountered, that all construction must immediately stop pending notification of authorities. It isn't talked about much, but there are a lot of instances every year where unknown graves or fossils are encountered during construction. In Arizona, if you find a grave, you must immediately stop and involve the State of Arizona Historic Preservation to properly document the remains and remove them.
I can only think of one instance where something hazardous was found on a construction site during construction. That was a gas tank, where every single official record indicated that the tanks had been removed 50+ years ago, but the contract found it within minutes of starting his work. In this case, he called us, and we got in tough with the State DEQ which had a program specifically for abating these old unknown tanks. Now this was a project that had already passed phase 1 and 2's with flying colors, hadn't been a gas station since World War 2, was used as a family home for decades. There was zero expectation it would be there. Thankfully the state came out, did their inspection, it was rendered safe (it still had gas in it), and the state paid for the abatement. The contractor ended up with a 2 week delay from grading work, but I allowed the other work to continue forward in other areas of the site so it didn't cost any time.
Kind of a long answer, but finding things after permits have been issued is exceedingly rare. I think in my career, I have overseen 50,000 permits at least and other than unknown graves and the one gas tank, that is all I have ever run into.
Now code enforcement running into dangerous things, that is a whole different story for another day!