The toxic aspect generally comes from the consequences of pursuing ambition by undermining others in a competitive context. If, hypothetically, the competitive pressure is off, then “ambition” is not nearly as likely to have negative consequences.

Here the person was saying we really need to actually put our minds to use, and that’s not really likely to be the ‘toxic ambition’ you fear.